A federal judge has ordered the Securities and Exchange Commission to rescind a controversial restriction on proxy proposals, handing a victory to activist shareholders who seek to challenge corporate policies on South Africa, environmental concerns and other issues.

In a U.S. District Court ruling, Judge June L. Green held that the SEC had failed to provide adequate notice before revising the proxy rule in August 1983. She said the agency's action "circumvents" federally required procedures for modifying regulations.

At issue is a rule dealing with attempts by shareholders to obtain new votes on previously defeated proposals. Under an older rule, shareholders could automatically get a second vote if a proposal received 3 percent in an initial vote. They could get a third vote if it drew 6 percent the second time.

Under the revised rule, the percentages were increased. Proposals had to win 5 percent the first time and 8 percent the second. Green ordered the old percentages reinstated. Under both rules, a proposal had to get 10 percent the third time to appear in later proxies.

"It's a big win for shareholder rights," said Cornish Hitchcock, a lawyer for the Public Citizen Litigation Group founded by consumer activist Ralph Nader, which filed the lawsuit in behalf of five church-related organizations with sizable stock holdings.

Linda Fienberg, associate general counsel for the SEC, said the agency had not decided whether to ask Green to reconsider her ruling, launch a new rule-making proceeding or appeal the order to the U.S. Court of Appeals here.

Green issued her order Friday and outlined her reasoning in a 12-page opinion yesterday. She said the SEC notice provided "not even a clue" as to whether it planned to raise, lower or keep the same percentages.