The Internal Revenue Service didn't exactly start out as an agency with artistic inclinations.

But as Uncle Sam began paying out increasingly hefty tax refunds to people who made charitable gifts of art objects, the IRS realized in the 1960s that it might pay to understand the alchemy of appraising art.

At the same time, the nation's art experts realized that inflated rogue appraisals--with attendant tax benefits to the owner--could prompt Congress to limit tax deductions for charitable donations. With less incentive for people to give art objects away, fewer art objects would be bought and donated: a disaster for museums and dealers.

As a result, the IRS and the art experts--dealers, museum officials and scholars--got together in 1968 and developed an elaborate machinery to police the practice of art appraisals. Any time an art object or collection valued at $20,000 or more is claimed as a charitable gift or included in an estate, in-house IRS experts and a panel of 12 expert advisers verify that the stated value reflects the price the object would command on the open market.

Over the years, they've had a chance to check the values of Old Masters paintings, Chinese sculpture, pre-Columbian art, African art, art nouveau, art deco, ancient glass, antique furniture, oriental rugs, antique guns, swords, an antique airplane, jewelry, stamps, fossils, a greenhouse and a collection of stuffed animals.

What they have found, not surprisingly, is that charitable donations tend to be overvalued--overall, by an average of a third, in the years for which statistics are available--and objects inherited by taxpayers tend to be undervalued. Average undervaluations have run as high as 60 percent (in 1972) and as low as 4 percent (in 1981).

Sometimes the IRS challenges result in dramatic revaluations; sometimes the courts and the experts agree with the taxpayers, and often the courts split the difference.

For example, in 1976, a ceramic vase made during China's Han dynasty about 1,900 years ago was donated to a public television station for its fund-raising auction. The station auctioned it for $360; the taxpayer claimed a charitable deduction of $50,000. A tax court ruled last year that the gift was worth $800; the taxpayer had to pay back taxes of $7,000, plus penalties and interest.

In 1974, a taxpayer claimed a $40,000 deduction for a mosaic table top donated to the archdiocese of Des Moines. The IRS said the mosaic was worth no more than the $12,500 the taxpayer paid for it. The tax court allowed the taxpayer a deduction of $26,500.

Sometimes, according to both members of the IRS' in-house experts and members of the art advisory panel, the government examines a valuation made by a charitable donor and increases the value. That sort of happy ending, however, means the cases don't come to court and the identities of the taxpayer and the artwork remain under the IRS' veil of confidentiality.

In practice, most of the government's judgments are made by a consensus of the advisory panel and the agency staff experts. The advisory panel, which meets three times a year, is not told whether the valuation it is reviewing was made in connection with a gift or an estate.

The IRS art experts gather photographs of the objects involved, documentation of recent sale prices for similar works, and other background material. Then the experts come to Washington, discuss the cases, and make their decisions.

"It's amazing to see how close the museum people, with a buyer attitude, and the dealers, with a seller's attitude, usually come to each other," said Wiley Grant of the IRS staff. Grant, like his colleague Karen Carolan, has a degree in art history.

The whole process offends some appraisers who feel that the calculators wielded by revenue agents do violence to the business of esthetic evaluation.

"It's an undemocratic way of handling things," said Richard Rush, an art market expert who has served as an adviser to taxpayers and as an expert witness in some tax cases. "They take a look at photographs, which may be inadequate. The experts ought to have access to the pictures or sculptures or carpets or whatever it is."

Carolan, however, counters that this criticism is unfair, since dealers themselves usually rely on photographs when they are negotiating transactions. "The art world couldn't work if they had to transport these works of art from place to place all the time," she said. "Using photographs is a common practice."

In response to the complaint that the IRS staff and its advisers do not have expertise on the vast variety of art they examine, she said: "It's not our job to be experts on the things themselves. It's our job to be researchers and examine the documentation of a valuation."

In that role, she frequently attends art auctions and stays in touch with museum officials and dealers to see what prices certain kinds of artworks are selling for.

The person who puts a true value on a work of art, says her boss, Geoff Taylor, "is the man who puts his bucks up."

Some art dealers credit the art advisory panel with stabilizing the once-chaotic world of appraisals.

"I think the fact there is such a panel has definitely had an effect on people who would always value things at three times their worth," said Gerald Stiebel, of the Rosenberg & Stiebel gallery in New York.