Moyer Smith, an advisory panel's top choice to succeed retiring Jim Kehoe as athletic director at the University of Maryland, has withdrawn his name from consideration for the job and it now appears likely it will be offered to Carl James, executive director of the Sugar Bowl.

Smith, associate athletic director at North Carolina, was on vacation and unavailable for comment yesterday. But Bill Cobey Jr., UNC's athletic director, said Smith told him Sunday he "didn't want to move his family up there."

James, 49, who resigned as athletic director at Duke University last year, was the advisory panel's No. 2 choice among four names it submitted to Robert L. Gluckstern, University chancellor.

John Toll, new Maryland president, is now acting on Gluckstern's recommendation. Sources said the university's Board of Regents, which makes the ultimate decision, could receive Toll's recommendation by the end of this week.

B. Herbert Brown, chairman of the Board of Regents and influential in athletic department matters, endorsed James yesterday.

"Carl James will get a lot of favorable consideration if he's recommended," Brown said. "I don't foresee any difficulties. But this is my personal opinion. I'm not speaking for (the Board of Regents) athletic committee, the board or the administration."

The board will convene a special meeting July 18, but Brown said he does not expect the athletic directorship to be considered at that time. He said, however, he will not hesitate to call another special meeting, so that the new athletic director can work with Kehoe before his Sept. 1 retirement. The board's next scheduled meeting is not until mid-August.

From his New Orleans office, James said yesterday, "It would be presumptuous to assume I would be offered the job . . . I have some good friends up there. I'm not suprprised they have interest in me."

James has met with Maryland officials at least twice. But he said he has not discussed contract terms with them. James indicated he would want a multiyear contract to persuade him to leave the Sugar Bowl.

He said he has two years remaining on his contract with the Sugar Bowl. He would not disclose his salary but said it was "equal to or better than those of athletic directors at major schools." (Kehoe's salary last season was approximately $44,000).

"If I made a change, it would not be based on economics alone, but on professional opportunity," James said. "I consider myself a professional. I'd be working in an area in which I have had experience.

"I would like to work more closely with student athletes than I have been here . . . It's up to them (Maryland) to evaluate where they want to go. I'm not going to get out and politic or write letters."

James would be the second Duke alumnus in Maryland's sports hirearchy. At Duke, where he played football and threw the discus, James was dorm counselor to a freshman basketball player, Charles G. (Lefty) Driesell.

From Duke, James went to work as a recruiter and sales manager for Atlantic Richfield Corp. He returned to his alma mater in 1954 as assistant athletic director and recruiter. He returned to private business in 1966, with Roadway Express Company. He returned to Duke as athletic director in 1970.

James denied reports that he had been forced to resign or was fired at Duke last year. "I had very few problems other than economic ones," he said.

James said he had recommended to the Duke administration that scholarship be dropped in nonrevenue sports, a plan that will be implemented this academic year.

"That," James said, "was not a matter of philosophy, but of economic necessity."

If he accepts the Maryland job, James will be faced with immediate financial problems inlight of plans announced over the weekend to reduce the College Park enrollment by approximately 3,000 students over the next four years.

That would slice $120,000 in mandatory athletic fees from Maryland's $3 million athletic department budget. The fee will be $40 per student this year and has been a controversial issue during the Kehoe regime.

Brown, the chairman of the board of regents; Pat Crossa., a spokeswoman for the chancellor, and James would not preclude the possibility that the fee would have to be raised to make up for the lost revenues.

James suggested raising more money from private sources and from increased football ticket sales as possible solutions. He added, "Nobody goes right in and says, 'Fine we lost some revenues. Therefore, we will have to increase fees.'"

"We have some fiscal problem ahead," said Brown. "If we want to maintain the programs we now have - meaning men's and women's - we'll have to generate more money. The funds will have to be generated or we'll have to have a retrenchment."

Brown said the university does not want to change its athletic emphasis or to operate athletics at a defifict. Maryland is among the 3 per cent of the nation's colleges and universities that the NCAA says operate their athletic program at a profit.