The Internal Revenue Service is searching for hundreds of Washington-area taxpayers, not to collect money from them but to give some back.

Altogether, the government is holding a total $1.1 million in unclaimed 1981 federal tax refunds for 2,599 taxpayers in the District of Columbia, Maryland and Virginia.

One missing Virginian has $16,700 coming from IRS, while one District of Columbia taxpayer is owed $12,247 and one Marylander $12,154, according to IRS representatives. Some other taxpayers on the IRS wanted list, however, are entitled only to $1 or $2 each.

IRS said checks were mailed out to each individual taxpayer but returned to IRS as undeliverable by the U. S. Postal Service, often because the taxpayers had moved with no forwarding address, or because the address was wrong on the form to begin with.

Nationally, the IRS has $29 million in unclaimed refunds for 62,560 taxpayers -- an average of $464 per taxpayer. The average unclaimed refund is $442 for the Maryland-District of Columbia district, where 1,550 taxpayers are eligible for $685,000. The average is $364 for Virginia, where 1,049 taxpayers are entitled to $381,681.

Unclaimed refunds represent less than a fraction of one percent of the refunds made, the IRS said. In the Maryland-D.C. area, for example, all but 1,550 of the 1.6 million refunds mailed to taxpayers were delivered, the agency said.

Taxpayers who are expecting a refund for taxes paid in 1981 or earlier should contact their local IRS office. The telephone number for Northern Virginia taxpayers to call is 557-9230; the number for D. C., Montgomery and Prince George's County is 488-3100, the number for the rest of Maryland is 1-800-492-0460.