The Supreme Court yesterday let stand a New Mexico judge's sweeping order compelling disclosure of a newspaper's confidential sources in a libel case.

The court also declined to consider a decision of a California court that in effect applied newspaper libel standards to a novel -- a work of fiction.

Both lower court decisions, in the view of many lawyers, went beyond anything sanctioned by the Supreme Court in freedom of speech and press cases.

The Supreme Court's unwillingness to accept either case for consideration sets no legal precedent and neither endorses nor condemns what the lower courts did. Both cases were denied without comment.

The justices may have declined to hear the New Mexico case because they considered it moot. The newspapers are no longer confronted with any contempt or default proceeding because the man suing them has since agreed to accept non-confidential material.

Judge Richard B. Traub, of a state district court in New Mexico, had told the Albuquerque Tribune to turn over anything pertaining to William C. Marchiondo, the plaintiff, it had collected over a five-year period.

Marchiondo, an Albuquerque lawyer, sued over articles, editorials and political advertisements run in the newspapers. One of the articles displayed a picture of Marchiondo below a headline saying "Organized Crime Showing Interest in New Mexico." One of the paid political ads included Marchiondo as among the "cronies" of New Mexico Gov. Jerry Apodaca who might "exert undue influence" on the governor.

To advance his suit, Marchiondo sought and recieved an order requiring the paper to answer questions about "all sources of information" for "every article published in your newspaper in which William C. Marchiondo's name was mentioned either directly or by innuendo."

The judge neither weighed the need for the information nor restricted his order to materials directly relevant to the articles and advertisements prompting the suit.

In that respect, the order was markedly different from procedures already upheld by the Supreme Court. In the Branzburg case in 1972, a grand jury considering criminal matters was judged to have a compelling need for specific information sought. More recently, the Supreme Court upheld an order to CBS producer Barry Lando, requiring him to explain his "state of mind" as he developed a "60 Minutes" story. The order did not seek disclosure of confidential sources.

In the California case, a state court had decreed $75,000 in damages -- including punitive damages -- against Doubleday and Company, Inc., and author Gwen Davis Mitchell for libeling a California psychologist in a novel called "Touching."

The novel, based in part on her observation of a nude therapy session conducted by the psychologist, Paul Bindrim, was judged to have defamed him by falsely describing his work in such a way that he could be identified, even though it used a fictitious name.

There was "actual malice," a California appeals court ruled, in part because Doubleday had been placed on notice by Bindrim before it sold the paperback rights that he was the therapist described in the novel and that Doubleday then "had a duty to investigate" the novel "for truth."

Such a standard is applied to newspapers publishing nonfiction. Applying it to a defamation case in a work of fiction, said an amicus brief filed by the Writers Guild of America and other literary organizations, "offends the First Amendment and will stifle creativity . . . If the novelist and her publisher here are held to answer, writers and novelists everywhere will be inhibited from drawing on their personal knowledge and insights into real persons. That is what novelists have done from time immemorial."