President Ford's choice for the chairmanship of the Republican National Committee dropped out of the race yesterday.

James A. Baker 11, who managed Ford's campaign after the Republican National Convention, said that he pulled out in the interest of party unity.

"It would be so much better if the new chairman could bring various elements together by his election, rather than take office at the end of a knock-down, drag-out fight," Baker said in a telephone interview.

He said that he withdraw although he thought "the situation was very winnable." By his count, he said. he led the seven-candidate field with about 50 votes.

Utah GOP Chairman Richard Richards has about 35 to 40 votes, and former Tennesse Sen, Bill Brock has about 35, Baker said. The 162-member committee will meet to vote Friday and Saturday in Washington.

Others disputed Baker's tally.

One GOP source said baker's tally "tested the water and found that the temperature wasn't right." He said that "all Ford's endorsement of Baker did was kill Richards' chances. The party has seen one Ford-Reagan fight, and it doesn't want another."

Richards has Ronald Reagan's tacit backing.

Backer, now in Houston, said, "I just thought that taking one name off the list might mean that the party won't be hopelessly split when it votes."

Another source said that to avoid a division, party members had responded to Baker's entry last week into the race by deciding to bypass both Baker and Richards "and go on to someone else."

Brock and Bob Carter, Washington public-relations man and holder of the powerless co-chairmanship of the GOP, seemed to be picking up strength yesterday.

The Senate Republican leaders Howard H. Baker Jr. (Tenn.) and Ted Stevens (Alaska), announced their support of Brock.

Sen. Barry Goldwater (Ariz.), speaking before Baker withdrew, said he opposed any effort by the President to dictate the choice of chairman, Goldwater said he supports Carter.

Traditionally, the national committee had been responsive to a Repulican President's choice.

Carter has been mentioned most often as a compromise choice if none of the other candidates could win a majority.

Other candidates are Buehl Berentson, former GOP Senatorial Committee executive director, Arthur Fletcher, presidential urban affairs assistant; and Thomas S. Milligan, Indiana GOP chairman.

Ford, Vice President Rockefeller, Reagan and John Connally met at the White House last week and discussed the GOP chairmanship, but Ford and Rockfeller failed to convince the others to back Baker.

Goldwater was miffed at not being invited to the White House session.

Baker had been something of a reluctant candidate. He said soon after the election that he wanted to return to his Houston law practice, and his family was known to be opposed to his becoming party chairman.

"I really don't think his gut was in it," one GOP official said. "If Ford's endorsement had flopped everyone over, and he could have on easily, then he would have done it."