Appeals court judges sharply questioned yesterday whether the Securities and Exchange Commission had a reasonable basis for adopting a controversial rule that requires hedge funds to register with the agency.

A divided SEC passed the rule in a 3 to 2 vote last year, citing evidence that the loosely regulated investment pools had become a breeding ground for fraud and trading abuses. But New York fund adviser Phillip Goldstein sued to stop the rule, arguing that the SEC had overstepped its authority and did not provide adequate foundation for the move.

Goldstein's case appeared to get a boost yesterday based on questions from two of the three judges on the U.S. Court of Appeals for the D.C. Circuit panel.

"You don't have authority to act simply because you exist," Judge Harry T. Edwards told Jacob H. Stillman, the SEC's lawyer.

A few moments later, Edwards said: "We have to test your thesis, and your thesis doesn't hold up."

Judge A. Raymond Randolph also expressed skepticism about the agency's arguments.

Legal experts cautioned that it is difficult to draw conclusions about how a court will rule based on questions asked by judges during oral arguments. The appeals court, however, has criticized the SEC's approach in a few recent cases.

Earlier this year, the court sent back for more research a rule mandating that mutual fund board chairmen be independent of management. The SEC retooled the rule, prompting a second, pending legal challenge by the U.S. Chamber of Commerce. That case is to be argued Jan. 6.

Last month, the court rejected a separate bid by agency lawyers to impose financial penalties on board members at an investment fund called the Rockies Fund Inc., ruling that the agency had levied the fines "arbitrarily and capriciously."

Former SEC Chairman William H. Donaldson made the hedge fund effort one of his central initiatives before he resigned in June. In recent years, the market has boomed to include more than 8,000 funds with over $1 trillion in assets. Average investors and pension funds increasingly are investing in the funds.

From 1999 to 2004, the agency filed 51 fraud cases involving hedge funds. Last week, Millennium Partners LP, a highflying New York fund, agreed to pay $180 million to settle trading abuse allegations lodged by the SEC and New York state Attorney General Eliot L. Spitzer. In September, two top officers at the Bayou Management fund pleaded guilty to criminal charges for engaging in a fraud that cost investors $450 million.

Stillman, the SEC's lawyer, stressed to the appeals court yesterday that the agency moved to register funds with more than 14 investors and $25 million under management to further its mission of protecting investors.

"Aren't they really getting at trying to enhance the government's ability to identify and prosecute fraud when it occurs?" Judge Thomas B. Griffith asked a lawyer for Goldstein. "That's really what's at the core of this."

The rule is set to take effect in February. Critics fear the hedge fund rule could foreshadow inspections and other efforts to rein in the funds. Before it was adopted, the plan had been criticized by Treasury Secretary John W. Snow and Federal Reserve Board Chairman Alan Greenspan, among others.

A ruling is expected within the next several months, according Philip D. Bartz, a lawyer at McKenna Long & Aldridge LLP who represents Goldstein.