The Reagan administration announced yesterday a new effort aimed at halting underpayments by oil companies and theft that deprive the federal government, states and Indian tribes of hundreds of millions of dollars each year.

Interior Secretary James G. Watt said the changes, many of them already in place, will bring the nation from $200 million to $650 million more annually in royalties the companies pay for the right to produce oil and gas on federal and Indian lands.

The collection of these royalties "is on the honor system," said David F. Linowes, chairman of a commission that Watt named to study the issue, and the system is not working.

The government has no way of verifying how much oil and gas is pumped, Linowes said. "Only a handful of audits have been conducted and they have revealed significant underpayments. Site security is deficient. Theft of oil is common," he said.

Watt said he favors all the recommendations of the Linowes commission, including establishment of a minerals management service in the Interior Department to monitor the royalties, which had been monitored by the scientifically oriented U.S. Geological Survey.

He said he favors raising the minimum royalty rate from 12 1/2 percent to 16 2/3 percent. Watt could raise the rate himself, but told reporters he will wait for legislative action.

Interior also will double its force of inspectors and take other measures to improve its checks on royalty payments and security on the leases.

President Reagan called it "unconscionable" that underpayments and theft have been going on so long when Watt and Linowes presented him with the report of what is formally called the Commission on Fiscal Accountability of the Nation's Energy Resources.

However, Watt and Linowes were reluctant to criticize the oil companies. They refused to name anyone who has been cheating the government and only after repeated questions did Linowes point out to reporters that his 267-page report names El Paso Natural Gas Co. and Conoco as companies against which claims were made for additional royalties after audits were completed.

He did not mention another section of the report that says that audits found that additional royalties should be paid by Texaco, Ocean, Mobil, Getty, Cabot, Sun, Amoco and Chevron.

Linowes said that Rep. Sidney Yates (D-Ill.) had told him that previous interior secretaries "did not have the political courage to move forward." But a minute later he praised the oil companies, saying he is confident they will do all they can to correct the situation. "If there was anything wrong, it was really a breakdown of procedural discipline," he said.

Watt said that "look-back audits" are under way with Exxon and Texaco to determine if any past royalties are due the government. He said there would be such audits at 25 oil companies, which pay 85 percent of the royalties.

Watt, however, was reticent about blaming the companies, concentrating his fire on "mismanagement" and the Interior Department. The monitoring of the system has been so poor that the commission was unable to determine exactly how much theft occurs. Nor could it discover the extent of underpayment or even how much underpayment is deliberate and how much inadvertent.

Watt and Linowes estimate only that underreporting is about 10 percent and they report that some security experts put oil field theft at between 3 and 6 percent.

The government expects to collect $5 billion in royalties this year and $14 billion by 1990. The rapid rise in energy prices has brought a sharp increase in royalties from the less than $500 million in 1971. In the first four decades of the lease operations, from 1920 to 1960, about $3 billion was collected.

There are 17,600 federal or Indian leases in 27 states.

Oil companies generally attribute the underpayments to mistakes, but Linowes said his commission found no example of an overpayment. Overpayments would be as common as underpayments if they all resulted from bookkeeping errors.

Charles J. DiBona, president of the American Petroleum Institute, said that he endorses many of the commission's findings, but "found no evidence to support the magnitude of possible underpayments or illegal acts that have been headlined to date in describing the results of the investigation."

"To imply, as some have done, there is a concerted industry practice to cheat the government out of royalties is erroneous," he said.