Forty-eight members of the House of Representatives from 21 states have formed a caucus to promote legislation affecting the arts.

The group's founder and chairman is Rep. Frederick Richmond, a Democrat from Brooklyn who was chairman of the board of Carnegie Hall for 16 years.

"The agenda is to get as many members [of the House] as possible to join," Richmond said yesterday by phone from Puerto Rico. "We have 48 without even sending out a 'Dear Colleague' letter. That's remarkable. Usually you have to send out dozens of letters."

Other members of the caucus, according to Richmond's office, include Reps. Morris Udall (D-Ariz.), Peter Rodino (D-N.J.), Charles Rangel (D-N.Y.), Toby Moffett (D-Conn.), Parren Mitchell (D-Md.), Geraldine Ferraro (D-N.Y.) and Margaret Heckler of Massachusetts, one of the few Republicans.James Jeffords, a Republican from Vermont, will be vice-chairman of the caucus.

"Mr. Richmond has asked him to recruit other Republican members," said Jeffords' legislative director, Margaret Downs. Jeffords, currently on Naval Reserve duty in the Persian Gulf, once worked on "arts and education issues," said Downs. "He tries to make sure rural areas are represented and get a piece of the action as far as arts programs are concerned."

Said Morris Udall yesterday from Arizona: "I thought I could maybe give the caucus some help. Basically the arts ought to be supported locally, but there is a place for a modest federal role."

Richmond and his staff have been organizing the caucus over the past three weeks through phone calls and conversations in hallways. No meetings have been held, but the caucus will send out a weekly newsletter to members of Congress telling them what arts legislation has been introduced and what's new with federal arts agencies.

"We hope the caucus will be a powerful force in passing legislation on the arts," Richmond said yesterday, noting several bills he has introduced. One is an "arts checkoff box bill" which "would let you check off any amount of money from your tax return that you wanted sent directly back to your county arts agency." Another bill allows artists to deduct the fair market value of their art work when they donate it to a museum or other institutional recipient.At present, they can only deduct the cost of materials. "But when you or I buy a painting or donate it," said Richmond, "we get a tax deduction. We want artists to have the same rights as we have." Richmond also has sponsored bills to aid artists with estate taxes and to institute proper labeling of toxic substances in art materials.

Richmond, who has been reelected for four straight terms despite a morals charge for sexual solicitation two years ago, is a millionaire collector of art and a pianist who once led the Navy band at Pearl Harbor. Some 20 years ago, he was one of the founders of the Carnegie Hall Corp. A member of the Agriculture and Small Business committees, he was in Puerto Rico yesterday studying food stamp problems.

Richmond recently criticized President-elect Ronald Reagan's choice of Richard Bishirjian, a professor at the College of New Rochelle, N.Y., as transition leader for the National Endowment for the Humanities. "I didn't think he had the credentials," Richmond said yesterday. "Reagan could have done better."

Notably absent from the caucus ranks is Rep. Sidney Yates (D.-Ill.) the influential chairman of the House interior subcommittee which approves the appropriations for the NEH. Yates declined the invitation to join as honorary chairman.

Although Yates reportedly supports the aims of the caucus, he said that he shouldn't join because of a possible "conflict of interest," according to Yates' aide, Mike Dorf. "What if he has to get mad at the arts during his hearings?" Dorf said. "I assume the arts caucus will be appearing before our committee in hearings." Yates was traveling to Chicago yesterday and was unavailable for comment.