A Virginia judge ordered state officials yesterday to give candidates and the public unprecedented access to working papers being used to tabulate votes in the unusually close U.S. Senate election.

The order by Richmond Circuit Court Judge James E. Sheffield will allow public scrutiny of a vote-counting process that the state's chief election officer freely acknowledges is fraught with potential errors.

Sheffield's order was sought by Democrat ANdrew P. Miller, who, according to an unofficial vote count, lost the Senate election to Republican John W. Warner by 5,190 votes out of 1.2 million cast. It was the closest general election in the short history of two-party competition in Virginia.

Separately Miller said yesterday he is not conceding the election to Warner although his lawsuit against the elections board refers to Warner as "apparently elected" and says that "Miller believes . . . (he) received the next higher number of votes . . ." in the contest.

"That's no concession," Miller said in an interview. He said he does not dispute the fact that Warner is the race's "apparent" victor, but said there is "clear precedent" for his hopes that the official canvass may erase Warner's current lead.

Neither candidate attended the court hearing, but shortly after it ended Warner appeared at the nearby Capitol for a brief meeting with Gov. John N. Dalton, who campaigned vigorously for the Republican candidate. Warner declined to comment on the vote counting process and continued to refrain from claiming victory until an official court is established.

After an hour-long hearing, Sheffield ordered the state Board of Elections to provide Miller with copies of tally sheets recording votes at the state's 1,857 precincts.

The tally sheets are one of two sets of documents used by the Board of Elections to determine the official vote total. Election Board Secretary Joan S. Mohan said in an interview yesterdayday that both sets of documents historically contain a high number of errors, errors that are caught by the state review of the election.

Referring to the 1977 gubernatorial election, Mahan said. "Last year, we found that 36 percent of the tally sheets contained at least one error and that 25 percent of the precinct abstracts contained errors."

The "abstracts are official vote totals compiled by city and county election officials and forwarded to the Board of Elections. They are used by the board in its official canvass - scheduled for Nov. 27 - to determined the final, official statewide vote tabulation.

Mahan said the Board of Elections inaugurated use of separate tally sheets to double-check abstracts in 1973 after she became aware that "our official vote totals in past elections contained errors."

Despite the acknowledged frequency of errors in the election documents, it was by no means clear yesterday that a closer scrutiny of the vote tabulation will reverse the unofficial election results.

Neither Miller nor Warner campaigns have alleged any irrefularities in the election. However, Miller's lawyer, former Attorney General Anthony F. Troy, told Sheffield the candidate must have immediate access to the tally sheets to have a meaningful opportunity to raise questions about precinct results before the Nov. 27 canvass makes the count official.

After election night confusion over the accuracy of the unofficial count by the News Election Service, that private vote tabulating service and the Associated Press conducted followup surveys of city and county election results. This double checking of unofficial returns has increased the Warner margin by about 600 votes.