Every other Thursday morning an increasingly troubled group of about 40 New Right conservatives gathers in a large, smoke-filled room on the second floor of a renovated stable eight blocks from the U.S. Capitol to plot strategy and exchange ideas.

The Library Court, as the group is called, is largely unknown to outsiders.

But the White House considers it important enough to send a representative to every meeting. So does Sen. Jesse Helms (R-N.C.).

Sens. Jeremiah Denton (R-Ala.) and Roger W. Jepsen (R-Iowa) have on occasion attended meetings. So have Rep. Jack Kemp (R-N.Y.), Health and Human Services Secretary Richard S. Schweicker, and Richard Wirthlin, President Reagan's pollster.

The reason is that the Library Court has become a farmers' market for New Right social issues.

It is a regular gathering point for a host of groups attempting to ban abortion, put prayer back in public schools, eliminate sex education, ban pornography and secure tax exemptions for segregated Christian schools and tuition tax credits for private schools.

Library Court takes its name from the Capitol Hill street where the group began meeting in 1979. Since that time, it claims to have played an influential part in stopping a host of liberal legislative and administrative initiatives.

But these are troubled days for the group. It is frustrated with the Reagan administration, and deeply divided over the abortion issue.

Most Library Court members consider Reagan one of their own. As a candidate, he embraced their issues and courted their leaders.

Library Court chairman Connie Marshner became a campaign adviser on family policy issues. At least two Library Court members, the Rev. Bob Billings, formerly executive director of the Moral Majority, and Jo-Ann Gasper, a self-described "pro-family leader" who formerly edited a magazine called "The Right Woman," received mid-level appointments to the new administration.

Gasper is HHS deputy assistant secretary for social services policy; Billings is in charge of regional liaison at the Education Department.

But as president, Reagan has disappointed the group. "We get the rhetoric, but we don't get the action," complains Marshner.

What upset the group most was Reagan's on-again-off-again support for tax exemptions for segregated Christian schools, an issue that first brought the religious right into politics in 1979.

When the Reagan administration initially lifted the ban on tax exemptions for such schools "we all applauded," says Bill Billings, Bob Billings' son and executive director of the National Christian Action Coalition, which represents 13,000 churches and 9,000 religious schools.

When Reagan proposed legislation to reinstate the ban "we all booed," adds Billings. "It was almost like he put the racial monkey on our back. Now we're being branded racists, which couldn't be further from the truth. We feel he has a moral obligation to reframe the issue. If he doesn't, he will lose the support of the religious right."

The split within the Library Court coalition over abortion is more complex, and evokes the kind of deep-seated emotions found only among true believers defending the morality of their cause.

It involves name-calling, backbiting, threats and scurrilous behind-the-scenes intrigue, which Marshner says add up to "a scandal of division" that threatens the future of the right-to-life movement.

Virtually everyone in the group wants to ban legalized abortions. The split is over the best legislative vehicle to accomplish that--a constitutional amendment sponsored by Sen. Orrin G. Hatch (R-Utah) or a human life bill sponsored by Helms.

Both were designed to appeal to pragmatists and began with the assumption that the current Congress would not pass a simple constitutional amendment to ban abortion.

Stephen H. Galebach, an attorney with the Washington law firm of Covington & Burling, conceived the Helms approach. It would ban abortion by declaring that human life begins at conception. Abortion would thus be murder.

Although this approach was widely condemned as unconstitutional, it held one great appeal for anti-abortionists: it needs only a simple majority for passage. An amendment takes two-thirds approval.

The Helms bill was the toast of the right-to-life movement for a few months, and was approved by a Senate subcommittee. Then came the Hatch amendment.

The idea of David O'Steen, executive director of Minnesota Citizens Concerned for Life, the amendment also was supposed to offer a winnable strategy for the pragmatists. It was written to answer two of the harshest criticisms of the Helms bill--that it would outlaw some contraceptive devices and that it would prohibit abortion exemptions for rape and incest.

The amendment would do so by giving states and Congress concurrent power to restrict and prohibit abortion. It also declares that "a right to abortion is not secured by the Constitution."

The amendment set off a vicious battle among foes of abortion. The National Conference of Catholic Bishops and several other groups quickly endorsed the approach. But other right-to-life groups condemned it as "sell-out" of principles with little chance of passage.

They received an unexpected boost when a memo written by Stephen Markam, a Hatch staff aide, came to light. The memo portrayed the amendment as a rather cynical political ploy "with a reasonable possibility of success on the Senate floor if everything comes together."

"There is also the advantage working for us that some senators may feel that they can cast a politically advantageous vote in support of the amendment with the knowledge that the measure will be defeated later by the House or by the states," the memo said.

Hatch opponents leaked the memo to the press, calling it a "smoking gun." Other leaks about internal divisions within the right-to-life movement popped up everywhere.

One group suggested that the U.S. Catholic Conference, which espouses liberal causes, had joined forces with the New Right, noting that Ernest Ohlhoff, executive director of the National Committee for a Human Life Amendment, which is supported by Catholic groups, had begun attending Library Court meetings.

In recent weeks, the battle escalated. Paul Brown, executive director of the Life Amendment Political Action Committee, sent a telegram to senators that said his group will consider any vote for the Hatch amendment "a total anti-life vote."

Marshner and Paul Weyrich, a leading New Right strategist, tried to put a stop to all this at a Library Court meeting last month. Both begged that right-to-life leaders unite behind Helms or Hatch. Weyrich said he was trying to save the groups "from destroying themselves." Marshner said the groups were confusing friends on Capitol Hill.

She warned: "The average politician will throw up his hands and say, 'A pox on both your houses--don't any of you ask me to do anything for you ever again.' If that happens, the right-to-life movement will have pulled the trigger on its own heart."

Judie Brown, president of American Life Lobby, was shocked. "This was something that never should have come up at Library Court," she says. "I cried right there in the Library Court meeting."

When she and other right-to-life leaders were asked to sign a "human life covenant" that declared "solidarity in this cause," Brown, who opposes the Hatch approach, refused.

"We know there are not 67 votes for anything on the Senate floor, and we do not want to be defeated this year," she says.

Those who favor legalized abortions couldn't be more pleased. They think the divisions in the right-to-life movement may keep any anti-abortion measure from being enacted this year.

"Any division on that side clearly helps us," says Nanette Falkenberg, executive director of the National Abortion Rights Action League. "The longer they stay split the more it helps us."