The Supreme Court ruled unanimously yesterday that a prosecution for obscenity must rest on standards laid down before - not after - a defendant engages in questioned conduct.

The defendants in the case were the Cinema X Theater in Newport Ky., its manager, its owner and its suppliers, American Amusement Co., Inc., and American News Co., Inc., of Durand, Mich. The films at issue included "Deep Throat" and "Swing High" and were shown before the end of February, 1973. At the time, the prevailing obscenity standard was that the Constiution protects expression unless it is "utterly without redeeming value."

Four months later, in June, 1973, court announced that it was toughening the standard to make the test of First Amendment protection "whether the work, taken as a whole lacks serious literary, artistic, political, or scientific value."

A federal judge instructed the jury to apply the 1973 standard retroactively. The injury - drawn from the large, heavily rural eastern Kentucky judicial district rather than from the Cincinnati metropolitan area of which Newport is a part - convicted all five defendants of conspiracy and four of them on seven substantive counts.

A divided Sixth U.S. Circuit Court of Appeals affirmed. The Justice Department argued that the ruling denied due process to the defendants.

Agreeing, Justice Lewis F. Powell Jr. said for the high court that the defendants, "engaged in the business of marketing dicey films, had no fair warning that their products might be subjected to the new standards." They were entitled to an acquittal unless the jury was directed to find the films "utterly without redeeming social value," he said.

Powell and four other justices voted to send the case back for a new trial. Four dissented, including Justice John Paul Stevens. In his first statement on obscenity since joining the court in December, 1975, Stevens said standards for obscenity prosecutions "are so intolerably vague that evenhanded enforcement of the law is a virtual impossibility."