Rejecting President Carter's proposed system for filling federal judgeships, Sen. Paul Sarbanes (D-Md.) has set up his own selection program, allowing some public participation but reserving for himself the right to make a final recommedation.

Sarbanes's decision was announced late last week, more than a year after Carter first proposed the establishment of a new selection system, giving the authority to make recommendations on judicial appointments to nonpartisan citizens' panels in each state.

The senator's decision provoked mixed reaction from the representatives of a coalition of Maryland organizations who had been pressing Sarbanes to name a citizen selection panel.

Representatives or two groups, the Legal Aid Bureau in Baltimore and the Maryland Commission for Women, denounced the Sarbanes plan, saving there is little difference between it and the old system. However, a spokesman for Common Cause of Maryland said Sarbanes's system was "a step in the right direction."

Two new U.S. District Court judgeships were created in Maryland as a result of legislation passed by Congress this fall. The legislation establishes 152 new federal judgeships nationwide.

Traditionally, federal judgeships, considered among the choicest of patronage plums, have been filled in each state by nominees chosen in secret by senators from the president's party. These individuals were then "recommended" to the president for actual nomination, but were rarely rejected.

President Carter has urged senators to name non-partisan citizen selection commissions to screen and recommend from three to five nominees for each judgeship. So far, senators from 18 states have set up such panels, but some others have been hostile to the idea.

Among those senators supporting the Carter administration's merit selection plan is Maryland's Republican senator, Charles McC. Mathias Jr., who will have no role in proposing nominees to the Democratic president. Mathias has been trying unsuccessfully for more than a year to get Sarbanes' cooperation in setting up such a panel.

Asked why he favored his procedure over a citizens' merit selection panel, Sarbanes said: "I don't see it [a panel] as meeting my constitutional responsibility of providing advice to the president."

Sarbanes' system will include these steps:

Persons seeking appointment to a federal judgeship must complete detailed questionnaires, available at the senator's Washington and Baltimore offices, by Dec. 15. Their names will remain confidential.

Sarbanes will meet with representatives of community groups, legal, business and labor organizations, and public interest and civic associations. He will request advice on the "qualities and capabilities" a federal judge should possess, and recommendations of persons who should be considered.

He will seek similar advice and recommendations from Maryland citizens, whom he is encouraging to contact him.

Sarbanes will conduct personal interviews with many of the applications - with all of them if this isfeasible.

Sarbanes will submit to the attorney general a recommended nominee for each judgeship.

Sarbanes said the procedure "will give a greater degree of broad public participation than the old system, while providing "the accountability which I think is important" in the process.

Sarbanes said he plans to "devote a significant amount of time" to his selection of nominees, and said he believes his prodecure will "compare favorably with what is being done elsewhere."

Thomas Keech of the Legal Aid Bureau in Baltimore, one of the coordinators of the Maryland merit-selection coalition, criticized Sarbanes's plan to keep confidential the names of those under consideration.

"How can public citizens groups comment on these people if they don't know who they are?" Keech asked. He said, however, the process "seems slightly better" than the traditional procedure.

Barbara Kreamer of the Maryland Commission for Women said she is "disappointed that Sarbanes, who had been encouraged on one side by the president and on the other by many Maryland citizens to name a citizen panel, had decided against that process."

The coalition had sought a panel that would include lawyers, laymen and groups "traditionally under-represented in the judicial selection process," Kreamer said.

Lawyer Derek Savage of the Maryland chapter of Common Cause said his group is interested in having a number of potential nominees for each judgeship recommended to the president, but added that Sarbanes's procedure is "obviously an improvement over the old system."