The Securities and Exchange Commission, working with the FBI and the Justice Department's organized crime task force, in what could be one of the biggest stock investment frauds in recent years, uncovered a scheme accused of defrauding high-rolling investors out of at least $50 million.

According to investigators, the scheme, which operated out of Miami, used greed and an exclusive aura to lure millionaire investors into a hush-hush, invitation-only investment company where the minimum investment was $100,000. Any questions investors may have had about the legitimacy of the funds were stilled by counterfeit computer printouts from reputable brokerage firms showing phony earnings of 80 percent to 90 percent on investments. In fact, the funds were losing money.

Investors could ask for daily accounts of their profits and take their money out of the funds at any point. The catch was that anyone who pulled out was out for good. Most investors stayed in, and those who opted out were apparently paid off with funds obtained from new investors.

The alleged scheme was apparently operated by Dennis Greenman, a broker with Barclay Financial Corp., a broker-dealer firm in North Miami. Last week the Atlanta regional SEC officials went to court and obtained a temporary restraining order shutting down Barclay, appointing a receiver and imposing a trust on assets of more than $25 million. From the time of the tip-off to the court hearing, SEC, Justice Department and FBI officials "worked around the clock for several weeks to produce enough evidence to secure the necessary court orders to shut down this massive fraud," said Bart Sacher of the SEC's Altanta regional enforcement staff.

The SEC charged that Barclay violated antifraud provisions of federal securities law and operated what amounted to an unregistered investment company during at least a two-year period.

The alleged fraud was uncovered by the FBI during an investigation of Tampa, Fla., land developer Allen Wolfson, an investor in one of the funds. When FBI agents subpoenaed Wolfson's bank records they traced money to Barclay.

Wolfson is a multimillionaire shopping center developer who was arrested in 1977 for bank fraud and accused of bribing officer, according to sources. The FBI has been looking into Wolfson's dealing since before then when he was involved in the bankruptcy of the Morristown & Erie Railroad, sources said.

Documents related to the investigation reveal a complicated, computerized scheme. "The psychology was great. This guy was very sophisticated," said one source. In fact, according to Chris Hoyer, special attorney with the Justice Department Strike Force on Organized Crime and Racketeering, investors were initially reluctant to talk to federal investigators because of their fear that they would be thrown out of the fund.

According to information from a variety of documents and sources, Greenman and a number of operatives in Florida and throughout the Southeast set up a number of limited partnerships or investment clubs which raised money from groups of individuals to invest through Greenman.

What he purported to do was to use a mysterious and extremely sophisticated computer program to take advantage of small differences in the price of the same stock in different markets to make money. For instance, a stock might be trading in one market for $5.25 for a few minutes while it was trading at $5 in another.

Greenman is said to have told investors that he would invest massive sums daily but take the cash out of the investments every night and invest it in treasury bills for still more money.

Investors, including millionaire businessmen, doctors and attorneys, had to have at least $100,000 to ante in and could only invest if they were recommended. One investor was Florida attorney Ed Rood, part of a group of investors that tried to buy the Tampa Bay Buccaneers football team and the financial backer for movies of live performances by Richard Pryor and Sammy Davis Jr. Another is C. Donald McLean, an attorney and a former member of the Jacksonville city council. Forty doctors from a Jacksonville hospital also invested.

Once in one of the 28 funds Greenman is said to have set up, they were urged to keep quiet about the investment on the grounds that if others found out how much money could be made it would dilute the earnings potential of the short-term trading fund.