The U.S. Postal Service earned a record surplus of $1.08 billion in fiscal 1982, twice the previous high, auditors reported yesterday, which means no increase in postal rates until well into 1984.

But the service's board of governors was not as pleased as might be expected. The figure, the board said, is a little too high, and the report was rejected. "There is a $106 million difference of opinion," said board member David E. Babcock, chairman of the audit committee.

For the first time, the Postal Service will need and receive no federal funding in 1983. It has had operating surpluses only twice before, $479 million in 1979 and $200 million in 1945, and between 1972 and 1980 got $920 million a year in government subsidies. There were deficits of $306 million in 1980 and $588 million in 1981, rolled back when the first-class postage rate went to 20 cents in November, 1981.

Postmaster General William F. Bolger has attributed the change not only to the rate hike but also to increased worker productivity, high mail volume, stable energy prices and slowed inflation. He said that $700 million of the surplus is from the operating budget.

Board member John R. McKean, an accountant, explained that the $106 million dispute between the board and its auditors, Ernst & Whinney of Washington, centered on calculations of workers' compensation payments that the Postal Service expects to make next year.

The Labor Department this year dropped 1,900 Postal Service cases without explanation from the 15,000 in its 1981 tally, McKean said, and the auditing company accepted that change.