The stock market started out yesterday with a bang, but ended the day with what was more like a whimper.

Ahead by more than 15 points after a feverish, record first-hour turnover on the New York Stock Exchange of more than 20 million shares, the stock market trimmed gains throughout the afternoon. The Dow Jones average of 30 industrials ended the day ahead 3.38 points to close at 886.87.

This followed a rise of 22.78 points on Wednesday, the best advance in more than three years.

Yesterday's close, however, still puts the Dow at its best level since last August. It is also the seventh gain out of the last eight trading sessions, for a net advance of 55.27 points.

Overall volume on the big board set a record of 65.5 million shares. The previous record of 63.5 million shares was set last April 17. The NYSE tape was running late most of the morning, and was behind by 22 minutes to one point.

Coming on top of Wednesday's turnover of 47.5 million shares, the volume performance was extraordinary because it occurred during what is usually one of the most sluggish trading periods of the year.

Despite the pullback - which analysts attributed to profit taking, some uneasiness on the part of investors at the disorderly pace of trading, and caution in advance of the weekly Federal Reserve money supply report - leading market watchers predicted the rally would continue through the end of August and lead the Dow above the 900 level.

"It will probably carry into September, and it seems the market should get to 930 and possibly do more," said Robert Farrell, vice president for market analysis at Merrill Lynch, Pierce, Fenner & Smith.

No specific news developments warranted the continued rally, other than the often expressed but somewhat tenuous view that interest rates may be close to their peak in the current cycle.

After the market closed, however, the fed reported that the basic money supply had gained a hefty $2.7 billion in the latest statement week, and the broader money supply had spurted $4.2 billion (details on Page E2).

And Saloman Brothers', chief economist, Henry Kaufman, issued a projection for credit demands through mid-1979 that said record credit flows will maintain upward pressure on interest rates which "will remain quite strong and will last some time into 1979."

Kaufman said the recent improvement in bond prices, which in turn has sent stock prices higher, "will be judged as a rally, and only a rally - not a basic reversal in the trend." The highly regarded money market economist projected that AAA utility bonds will go to 10 percent or higher, and that the prime rate will be at 9.5 to 9.75 percent by year-end. He admitted that, for the time being, his was "a minority view."

Stock analysts said most of the impetus for the market's current rally was coming from highly liquid institutional investors like pension funds and bank trust departments who aggressively are committing their massive cash reserves to equities in the belief that rates may have hit a top.

"Everybody is a trend follower, and the institutions watch the action and are pretty convinced the trend is up," said Michael Metz, a vice president with Oppenheimer & Co. "They don't know why, but they try to rationalize it by saying that interest rates are peaking, that the dollar has fallen so fast that it's got to rally, and the Fed has taken a stance where it really won't bring on a credit crunch. So it's a favorable environment.

"Remember, the amount of money available is literally tens and tens of billions of dollars," Metz added.

Larry Wachtel, vice president with Bache Halsey Stuart Shields said "the cutting edge in the spring rally was the foreign investor, but the cutting edge in this rally is the domestic institution."

United Press International reported the following:

The NYSE common stock index rose 0.31 point to 58.13, and the price of an average share added 17 cents. Advances outdistanced declines by 1.004 to 626 among the 1.945 issues traded at 4 p.m. EDT.

Composite volume of NYSE issues listed on all U.S. exchanges and over the counter totaled 68,359.420 shares, up from 52,153.340 shares Wednesday.

The American Stock Exchange index gained 0.68 point to 157.26, a 1978 high, and the price of a share climbed 6 cnets.

The National Association of Securities Dealers' NASDAQ composite index of OTC stocks added 0.86 point to 129.02, another high for the year.

At the 4 p.m. close, Citicorp was the most active NYSE-listed issue, up 1 1/4 to 26 1/8 following the sale of a block of 105,000 shares at 26 1/4. Some analysts said they thought the stock was helped by speculation interest rates had crested.

Westinghouse Electric was the second post active issue, up 3/8 to 24 5/8. Boeing, followed, off 1/2 to 70 1/8. Boeing has received a $156 million order from Thai Airways International for three 747-200B jets.

Blue-chip RCA rose 1/4 to 29 7/8 in trading that included a block of 110,000 shares at 30 1/4.

American Telephone & Telegraph, the nation's most widely held stock, gained 1 1/4 to 61 5/8 in active trading.

International Business Machines retreated 5 to 285. IBM and Xerox earlier this week agreed to end 12 suits involving patent disputes, and IBM consented to pay Xerox $25 million. Xerox picked up 5/8 to 60 5/8.

Merrill Lynch, Pierce, Fenner & Smith added 3/4 to 21 1/4. The nation's largest brokerage had record second-quarter earnings because of the spring rally and stands to benefit from new high volume.

Among the other brokerage issues, E.F. Hutton gained 1/2 to 18 1/2 and Dreyfus Corp, 7/8 to 14 5/8.

Tropicana Products spurted 1 1/4 to 52 and Beatrice Foods tacked on 1/2 to 26 1/8. The Federal Trade Commission renewed its attempt to block Beattrice Foods' acquisition of Tropicana Products.

OKC Corp. plunged 3 1/4 to 17. Saudi Arabian businessman Ghaith Pharaon said Wednesday night he "has determined not to proceed at the present time" with a proposed tender offer for OKC because the firm is under investigation by a regional office of the Securities and Exchange Commission.

On the Amex, advances topped declines by a 3-to-2 margin among the 929 issues traded at 4 p.m. Volume at that time totaled 6,640,000 shares compared with 5,340,000 Wednesday.