An actor, a director and an arts agency representative had harsh words yesterday for the Reagan administration's proposed tax changes, saying they would damage the performing arts.

Their remarks hardly brought down the House. A handful of congressmen (chairman Dan Rostenkowski [D-Ill.] was absent) and about 80 spectators attended the Ways and Means Committee hearing on how the comprehensive tax overhaul would affect charitable giving.

Tony Randall, best known for playing Felix Unger in the TV series "The Odd Couple," began by applauding the "enormous explosion" of the arts during his lifetime. The 65-year-old actor cited his home town of Tulsa, now host to an opera company, two museums and a ballet troupe. None existed when he was a child, he said. This proliferation is due to "direct and indirect support for the arts," he continued, referring to tax breaks for those who donate to arts institutions.

Stage director Des McAnuff told members of Congress at the Longworth building that the performing arts, which already have "slim" operating budgets, cannot afford to lose that benefit; and he spoke of the importance of continuing the business deduction for entertainment.

"It's impossible to put a price on art in our society," said McAnuff, who directed "Big River," which won seven Tony Awards this year, including Best Musical.

Bernadette Nolan, secretary of the Illinois Arts Council, protested the proposed elimination of the entertainment deduction. Ending that, she said, could "threaten the vitality of the performing arts," since, according to her statistics, 30 to 35 percent of all theater tickets are purchased as a business expense. She said ticket prices, already high because of rising production costs, would rise still more if such support were withdrawn.

The Reagan tax plan would cut tax rates for individuals, but also would eliminate a number of popular deductions. In the case of entertainment, the Treasury Department has maintained that deductible tickets have inflated prices for everyone.

Nolan argued that taking clients to the theater is as legitimate a business expense as office furniture or "printing matchbooks with the company's logo."

Randall closed by quoting from a letter he had received from the executive director of the Los Angeles Philharmonic Orchestra: "Government funding is on the decrease. Tax incentives for charitable giving are being whittled away, and claims on what remains of the charitable dollar have increased dramatically."

After Randall finished, Rep. Thomas J. Downey (D-N.Y.) said, "The State of New York subsidizes musicians and artists. Is it fair to say that this is a valuable way to spend our money as states?"

Replied Randall, "That's a very difficult question because, as my colleague Mr. McAnuff has said, no one has been able to put a real value on art. Is it important to train young people to go into the arts? I think it's the most important thing of all."