Millions of Americans are filing out their income-tax returns this week and simultaneously grumbling about the inequities of the system. But horseplayers have more cause to grumble than most any other segment of the population.

It is painful enough to pay taxes on our income. Many thousands of horseplayers, however, will be paying taxes on nonincome, taxes that they do not legitimately owe but which the system wrings out of them, anyway.

The source of this injustice was the 1977 legislation that required a 20 percent withholding tax on certain racetrack payoffs of $1,000 or more. This, of course, was blatantly discriminatory. Any congressman who voted for a comparable withholding tax on, say, stock-market transactions, would have been practicing law back in Peoria after the next election.

This legislation affected great numbers of bettors - not just a handful of big winners - because exacta and triple wagering have proliferated at the nation's racetracks and $1,000 payoffs on them are commonplace. A typical active horseplayer probably hit one such payoff in 1978. And, being typical, he probably wound up the year losing money, anyway.

Because he must pay taxes only on his net gambling profits for the year, he is entitled to a refund of the money the IRS withheld. But when he tells this to the IRS, claiming losses that offset his winnings, the IRS will probably audit his tax return and tell him to prove those losses.

An that is the rub. "You have to go through so many gyrations to prove your losses that very few people go to the trouble," said Arnold Kirkpatrick, director of research for the American Horse Council.

"The IRS wants the bettor to keep a diary of all his gambling activities, including the names and addresses of people he went to the track with each day, a record of all the horses he bet and the amount of the payoffs, and all his losing tickets. The tickets had better be reasonably sequential, with no heel marks. Because of the complexities of substantiating the losses, I'd say a very low percentage of people ever get their money back."

Even the rare horseplayer who does keep thorough records may run into other problems. According to the IRS, a gambler must list all winnings as income and offsetting losses as itemized deductions.

The 80 percent of taxpayers who take the standard deduction, and do not itemize, are either out of luck or forced to suffer serious inequities. A reader, James Downey of Silver Spring, submitted a hypothetical example of the unfairness that this tax law can create:

Mr. X. is married and earns $18,000 in wages. He has only $2,000 of allowable deductions, and so he normally elects to take the standard deduction of $3,200.

During 1978, he hits a $1,200 exacta, but still suffers a net loss for the year and has solid proof of his loss. When he computes his taxes for the year, his gross income now becomes $19,200, and he claims $1,200 as an itemized Schedule A deduction.

Now his total itemized deductions of $3,200 equal the $3,200 standard deduction to which he is entitled. But his income is now $19,200, inflated by $1,200 of "winnings" that he did not win. So he is still paying taxes on nonexistent income.

Other types of unfairness abound in the tax laws' treatment of gamblers. For example, almost everyone engaged in an activity conducted for profit can carry over losses from one year to another.

A soybean speculator who lost $50,000 in 1977 and earned $50,000 in 1978 has broken even, as far as the government is concerned.

But if a gambler loses $50,000 in 1977 and wins $50,000 in 1978, the winnings are taxable and the losses are tough luck. (Besides gambling, there is only one other category of activity in which the carry-over of losses is not permitted: crime)

Such discrimination is not likely to be redressed. The IRS can hardly be expected to start showing compassion for horseplayers (or anybody else, for that matter) and few congressmen are inclined to vote for measures that can be perceived as taxes breaks for gamblers.

The American Horse Council is lobbying to raise the threshold for withholding-tax payoffs from $1,000 to $5,000, and Kirkpatrick said he thinks the prospects for such legislation are excellent. That would only reduce the number of instances in which horseplayers have their pockets picked by Uncle Sam, but that is the best we can reasonably hope for.