White House chief of staff James A. Baker III said yesterday "it is my best recollection" that William J. Casey was the senior official in Ronald Reagan's 1980 campaign who produced a briefing book from President Carter's reelection team before the nationally televised debate between the two candidates.

Casey, who was campaign manager for Reagan and is now CIA director, said he had "no recollection" of the document.

Their comments are in letters to Rep. Donald J. Albosta (D-Mich.), a House subcommittee chairman who is looking into the briefing book episode. Two other Reagan administration officials, Office of Management and Budget Director David A. Stockman and White House communications director David R. Gergen, also said in response to questions from Albosta that they knew of the Carter briefing material.

Stockman said it was "useful" in preparing Reagan for the debate. Gergen said that he never saw it but that others told him it was not "important or dramatic."

Meanwhile, Myles Martel, a Villanova, Pa., consultant who worked for the Reagan campaign, said in a telephone interview yesterday that he urged another campaign aide, Frank Hodsoll, to return the Carter briefing materials at the time. "My impression was we had something that we shouldn't have had," Martel said. "It should have been returned."

Martel said Hodsoll "didn't debate me" and responded "with something to the effect that I had a very interesting position on the subject."

Hodsoll, who is now chairman of the National Endowment for the Arts, said yesterday he does not recall any discussion with Martel on the ethics of having the material but said he might have talked about it "in passing."

Hodsoll has said previously that he had been given several hundred pages of question-and-answer briefing material from the Carter campaign but never knew the origin of it. Martel said he was hired by the Reagan campaign to prepare Reagan for the Oct. 28 debate with Carter and an earlier debate with John B. Anderson.

He said he never saw the Carter briefing material, but urged that it be sent back when Hodsoll told him about it Oct. 26 at a dinner at Hodsoll's home. Martel said the material was "inconsequential" and he did not mention the episode in a book he subsequently wrote about presidential debates.

In their letters yesterday, the Reagan administration officials said they had not made an effort to determine the source of the Carter briefing materials. Those who remember the documents said they were position papers and not strategy memos.

Baker said he has "no recollection that I ever received or saw any debate strategy or sensitive debating points prepared by the Carter campaign or Carter White House." But Baker said, "I do remember briefly seeing a large loose-leaf bound book I believe in a black binder that was thought to have been given to the Reagan camp by someone with the Carter campaign."

Baker recalled that Casey gave it to him "with the suggestion that it might be of use to the Reagan Debate Briefing Team" headed by Gergen and Hodsoll. After thumbing through it, Baker said, he "passed it on" to the team.

The chief of staff said he "never, directly or indirectly" asked anyone to get the Carter briefing material, which he described as a "compendium" of Carter administration policy positions. Baker said he never tried to determine the source.

"There was nothing on its face that suggested it may have been an official document or a document sufficiently sensitive to have been controlled or closely held," Baker said. "In any case, I would submit that, unfortunately, it is not uncommon in campaigns--just as it is not uncommon in the ordinary business of the press and the Congress--for such material to be given or sent . . . . "

Baker said "there is obviously high political content" in efforts to make an issue of the briefing book episode.

Baker and Gergen said they are not aware of any evidence that the briefing book was illegally obtained. That charge was raised recently in a new book, "Gambling with History" by Laurence I. Barrett, who said the material was "apparently . . . filched" from the Carter camp. He did not identify the source of the material.

In the letters sent to Albosta, there appeared to be a contradiction between Baker and Casey. Baker recalled that Casey gave him the book. Casey wrote that he had "no recollection" of any papers setting out Carter's debate strategy or debating points. And Casey said that his secretary, two special assistants and five deputy campaign directors had told him they have no knowledge of briefing material "resembling that" described by Barrett.

But in his letter Casey did not address the "black book" described by Baker. In a statement subsequently issued yesterday, Casey said he did not remember "any black book," but added that he sent Baker all papers that crossed his desk concerning the debate. Casey did not specifically rule out the possibility that the Carter briefing material was sent to Baker.

Gergen said that he does not remember "ever receiving or seeing" the debate briefing book, although he may have seen "some pages." He said his judgment that it was not important was based on what others told him about the document and the low-key way it was used.

While Gergen and other White House officials have claimed the material was of little help in readying Reagan for the debate, Stockman said it was "useful" in "a very particular and unique sense." He said the documents helped him prepare for his stand-in role in the practice debate with Reagan, but that the material would not have helped others in the campaign.

Stockman said the material--"a thick, unbound set of pages"--was delivered to his congressional office Oct. 23, 1980, the day before debate preparation began with Reagan at his campaign estate, Wexford, in Virginia's hunt country. Stockman said the documents "dealt entirely with policy issues . . . . "

A White House spokesman said Reagan had read the letters and told his aides to cooperate with Albosta. The congressman said he would review the letters "thoroughly."