Since Black Monday, large numbers of small investors, nursing their wounds, have been staying out of the market. And some big Wall Street brokerage houses, seeing their profits from the ''retail trade'' erode, have gotten the message.

Stunned by the wild gyrations of the markets, especially the 508-point collapse of the Dow Jones index last Oct. 19, many investors have wised up. The Brady commission report showed how a tidal wave of selling initiated by big operators cascaded over their small investments.

Fearful of government regulation, some Wall Street firms now are trying to win back the confidence of small investors by reducing the amount of computer-driven program trading they do for their own accounts. It's an encouraging start, but the question remains whether they've gone far enough.

But if Wall Street is trying in a limited way to take some of the sheer gamble and glitz out of the casino -- and thereby lure back the small investor -- the federal government hasn't exactly leaped to put the Brady commission's recommendations into practice.

Shearson Lehman Brothers took the lead in voluntarily giving up program trading and was quickly followed by a handful of other brokerage houses. This displayed a sensitivity to the fact that the process of combining offsetting purchases and sales of stocks and stock-index futures was driving the market up or down like a roller coaster, with little relationship to the real value of individual stocks.

''There is a high degree of agitation and unhappiness on the part of everyone in the investment community because of the volatility of the markets,'' said Peter A. Cohen, chairman of Shearson. ''And we are now convinced that some of that volatility is created or exacerbated by the effects of program trading on the marketplace.'' But remember: the pullback by Shearson and others applies only to program trading for their own accounts. They will still do it for their big customers.

Meanwhile, the New York Stock Exchange decided to try a trading halt for 10 minutes or so in individual stocks if their prices rise or fall by an extraordinary amount. And the Big Board will bar the use of its automated trading system for computer programs if the Dow Jones index rises or falls 75 points or more in a day.

These actions by the Big Board are eminently sensible examples of the kind of ''circuit breakers'' recommended by Brady to give the exchange a chance to stabilize when under pressure.

Some free-market purists object to any interference that blocks or delays the normal buy/sell flow. But the Brady report shows that the events of October were not normal, any more than the panicky rush of a theater crowd to fire exits is normal.

Says a highly placed government regulator:

''Brady's argument -- which is clearly one that must be addressed -- is that the circuits will break one way or the other, if you get a surge of transaction demand in a system whose capacity is limited. And his argument, essentially, is that you might as well choose the form in which the breaks occur, rather than having it done ad hoc . . . which means a total breaking down of trading.''

An absolute necessity, requiring government action, is a boost in margin requirements for dealing in stock-index futures on the Chicago exchange, closer to the 50 percent established by the Federal Reserve Board for buying the underlying stocks. The voluntary increase put in by the exchange since Black Monday boosts the margin only to 10 percent from the 5 percent at the time of the crash.

Also, there must be more effective overall regulation of the markets, whether Wall Street welcomes that or not. The Brady commission's suggestion that the Fed be put in charge isn't welcomed at the central bank. Officials are uncomfortable with the idea because, among other reasons, they fear it would make an already powerful agency too powerful. Presumably, the Fed feels that the same agency that is looked on as lender of last resort should not be the main regulator of financial markets.

Nevertheless, Felix Rohatyn of Lazard Freres suggests that the Fed is the logical place to consolidate authority over margins (including those on options and futures) and over capital requirements, while the Securities and Exchange Commission should exercise all other regulatory functions, on futures and options as well as stocks.

Some in Wall Street would not be happy with expanded SEC control, because, they argue, the SEC doesn't really understand all its problems. (Only recently, the SEC gave program trading a clean bill of health.)

But in light of what happened Oct. 19, somebody has to do a better regulatory job. Whatever the Fed's reluctance, it will do what Congress tells it to do. The risks inherent in adding to the Fed's powers may be smaller than allowing rudderless markets to generate chaos -- and, maybe, another Black Monday.