A federal judge in San Francisco yesterday refused to let prosecutors in the espionage trial of Jerry Alfred Whitworth use a series of letters Whitworth allegedly wrote to the FBI last year offering help in breaking up a major spy ring.

U.S. District Judge John P. Vukasin said prosecutors had so far failed to prove the letters, signed "RUS," had been written by Whitworth. He called the letters "in essence confessions," and said if the government could prove Whitworth wrote them, there would be virtually "no need for a trial."

Whitworth, 46, the sole remaining defendant in the Walker spy ring, is to stand trial Feb. 10 on 13 counts of espionage and tax violations. A retired Navy communications specialist, he is charged with passing classified documents to confessed spy John Anthony Walker Jr., including cryptographic "key lists and key cards" used to encode and decode sensitive information.

Vukasin said he would permit the government to introduce at the trial a "Dear Johnnie" letter from Whitworth to Walker, and a "Dear Friend" letter from Walker to his Soviet contact, in which the government contends Walker referred to Whitworth by the code name "D."

Vukasin said he would reconsider his decision on the "RUS" letters if the government could demonstrate Whitworth's authorship. Assistant U.S. Attorney William Farmer said the government would try to show Whitworth wrote the letters by analyzing their writing style.

In the first letter, dated May 7, 1984, and signed "RUS, Somewhere, USA," the author "states . . . he has passed top secret cryptographic key lists for military communications . . . to his contact," who funneled the information to the Soviet Union, acording to an FBI affidavit.

The letter, addressed to the FBI's San Francisco office, was postmarked Sacramento, Calif., not far from Whitworth's home in Davis, Calif.

In the second letter, dated May 21, "RUS" said his contact had been involved in espionage for more than 20 years and "plans to continue indefinitely." John Walker admitted spying for the Soviet from 1968 until his arrest May 20.

The third letter, dated Aug. 13, 1984, said "RUS" had decided "it would be best to give up on the idea" of exposing the spy ring.

Whitworth's lawyer, Tony Tamburello, denied that his client wrote the "RUS" letters and said the government had been unable to prove that contention despite a battery of tests trying to match typewriter balls, watermarks on the paper and envelopes, and fingerprints. "None of it points to Jerry Whitworth," he said.