It bears a staid name -- the Chinese Gold and Silver Exchange Society -- but at first glance it looks like a riotous outing of a young men's drinking club.

In the rectangular pit, traders casually dressed in sports shirts and slacks or jeans shout, gently shove each other, and wave their arms in a kind of gleeful bedlam.

An outward wave of the hand means the trader is selling; an inward wave means he is buying. Extended fingers signal his price in Hong Kong dollars. A slap on the back means the deal is made, and a small fortune in gold changes hands.

The appearance of cheerful chaos is deceiving because the Hong Kong gold market does some of the world's most serious business. Several hundred million dollars' worth of gold changes hands each trading day. It is the world's second or third largest gold exchange -- New York's is first and London's does not publish trading figures.

For most of the trading day here, while New York and London sleep, Hong Kong is the world's most flourishing market, dwarfing the others in Asia. For Arab millionaires, Swiss and West German banks and London bullion dealers, Hong Kong is where the action is during those hours.

The market has grown to international prominence in recent years, but its importance primarily still rests on the ancient Chinese affection for gold as the safest of havens in times of wars, insurrections and other calamities that have wracked Asia this century.

"Chinese have a traditional affinity for gold," observes Joseph S. K. Lo of International Gold Corp. Ltd., which markets the South African krugerrands here. "They've used it over the centuries for investments and hedges against inflation . . . . Maids, servants, the well-to-do -- somehow they all buy gold bars."

That affinity extends throughout Southeast Asia, where Chinese have left their marks. In the 1960s and early 1970s, the elites in South Vietnam and Cambodia stocked up on Hong Kong's gold as a hedge against a communist takeover. Thousands of those gold bars have since turned up in the hands of Vietnam's new rulers, who demanded them as bribes from fleeing refugees, the "boat people."

The Hong Kong Chinese Gold and Silver Exchange Society (silver is no longer traded) was born in 1904, the creation of Chinese bankers who by and large were of the second generation of Chinese immigrants who struck it rich in this British colony.

By 1910 the society had its own constitution and bylaws, which restricted membership to Chinese. Its 194 current members are all Chinese, despite some pressures to open the ranks to U.S. and European dealers. Foreigners now must deal through one of the local Chinese members.

Woo Hon Fai, the gold dealer and entrepeneur in real estate who is the exchange's current president, attributes its recent successes to its members' traditional expertise in handling gold. "They have long experience in the gold trade," he said.

But he also acknowledges that Hong Kong's favorable business climate is a key element. There are no capital-gains taxes as in the United States, and no value-added taxes as in Britain. There also are no currency-exchange regulations, which means the speculator who cleans up in gold can leave for home with his winnings and no questions asked.

Some of the gold sold here is taken by dealers and goldsmiths, who convert it into expensive ornaments and jewelry, and some is bought by the general public for storage against hard times.

But by far the biggest share is traded by speculators hoping for either short- or long-term profit. One expert source estimates that only about 2 percent of the exchange transactions actually involve the physical delivery of gold bars. The rest are paper transactions in which no gold physically changes hands.

At one time, Arabs awash with oil money invested here, but the exchange members now report those investments have largely dried up since the price of gold fell sharply last year.

After shooting up in feverish trading to about $850 per ounce in January 1980, the price of gold has fallen to about $400, and many authorities believe it will go down even further.

The price rose swiftly before because of high inflation rates and the decline in the value of the U.S. dollar. The Reagan administration's determination to curb inflation with a tight monetary policy and high interest rates sent big money scurrying from gold into dollars and the price plunge for gold sent the market here sharply down.

But with the price hovering around $400, gold again looks good to Hong Kong buyers. More gold is being bought by small buyers here these days, although the total volume on the Hong Kong exchange this year is about 20 percent lower than the average daily volume last year.

"I was in New York last March and nobody was buying gold -- nobody," said one dealer here. "Back here in Hong Kong they were lined up to buy gold from the banks. They have a totally different attitude here toward gold.

"There are more Americans and Europeans selling gold here now," he added. "You won't see people here stop buying gold."

To some international experts, the Chinese exchange here seems a bit stodgy, if not anachronistic. It still deals in gold taels, an ancient Chinese weight, instead of the Western ounce. It also denominates gold sales in Hong Kong dollars instead of U.S. dollars.

Critics argue that the local market would become even bigger if it internationalized itself by dealing in ounces and U.S. dollars and possibly by admitting a few Western dealers.

The 194 Chinese members have kicked these ideas around but so far have determined to keep things as they were established by their ancestors more than 70 years ago.

Who wants to change a good thing? As President Woo puts it, "We are active and we are prosperous."