OUR SOCIETIES are today being asked to devote too many of their resources to the relief of the elderly and the handicapped. It is difficult to say so, of course, without being abused. One is insensitive, it is said, uncompassionate and cruel. The more sentimentally liberal of one's friends -- who are easily convinced that they alone have kind feelings -- raise their hands in pious unbelief. "You can't really think that," they say; but one has thought hard about it.

One knows that one is going to be accused of kicking the paralyzed, of being little better than hoodlums who tip over wheelchairs in the streets, of being the kind of person who would not help an old lady across the street. Yet one is not entirely alone. Even liberals who are not satisfied with mere sentimentality agree with one's doubts, but they whisper their assent only in dark corners where they will not be overheard.

At the most obvious level, one is asking how much a transit authority or a university or a library or any welfare agency should be expected to spend in removing the inconveniences of the elderly and the handicapped, at an unavoidable cost to others. The Congressional Budget Office has estimated that final regulations issued by the Department of Transportation a year ago would be "very expensive" for the "relatively few" who would benefit: an expenditure of $7 billion over the next 30 years to give fewer than 15,000 wheelchair users access to buses and some 2,000 to subways. Each bus or subway trip would cost $38, whereas if there were a better provision of taxis and special automobiles it would cost $7.

The tight budgets of some institutions have been stretched and distorted in recent years by the physical changes which they have had to make to their buildings and the services they must provide. If the full plan for transportation were put into effect, the mayors of cities such as New York, Chicago and Philadelphia say that it would finally bankrupt their rapid transit systems and their cities.

But one is also asking about more than the cost. How far can the idea of citizenship go on being widened until it means that any self-promoting group can claim the right to have its inconveniences removed at whatever cost?

Such rights too easily become indistinguishable from privileges. When I was in Wisconsin last month, Gov. Lee Dreyfus was proposing that, since the elderly do not go to school and college, they should be exempt from taxes for education. An article in The Capital Times in Madison described this proposal as "startling and disturbing." In fact it is preposterous. But it is well in line with many of the assumptions which are made today about the rights of individuals. Before long it may be offered as a constitutional amendment.

For any group may now organize itself to bring pressure to bear on the point of leverage in the political system. Rep. Bob Duncan of Oregon was defeated in the Democratic primary two weeks ago by Ron Wyden, whom The Washington Post described as "a 31-year-old activist for the elderly." As chairman of the transportation subcommittee of the House Appropriations Committee, Duncan has had the responsibility and courage to ask for some delay in implementing the Department of Transportation's proposals, at least until further studies are made. Wyden, in response, helped to organize the state's Gray Panthers -- a revealing enough name -- and "his campaign made use of several hundred elderly volunteers." The elderly are a useful army to enroll. They have time on their hands, while the producers and earners are working. Wyden described his victory as "a triumph of grass roots politics," and this is indeed the claim now made for particular groups which are seeking privileges.

It is because the case of the handicapped and the elderly is a hard one that we must be candid about it. For hard cases do not make good law. Apart from the tangible resources which our society has to find, there are also such intangible resources as time and attention. A wholly disproportionate amount of time is now given on television to programs or news items about the elderly and the handicapped. Of course they appeal to our good feelings, just as do photographs of wistful children and the gnarled old, two of the favorite (because simple) subjects of photography. But we are in danger of reserving our sympathy only for such "minorities" and having none left for the daily lot of the "majority."

It is my experience that this majority of reasonably fit people who are the workers and earners are growing more and more tired of having their feelings exploited on behalf of the numberless minorities who can claim to be disadvantaged. We acknowledge that the "middle Americans" have been neglected in recent years. But now it is these same people who must again sacrifice.

It is often said that our "consciousness has been raised" -- that phrase which covers so much sloppy good feeling -- by bringing the handicapped and the old "out of the closet" -- another phrase which is used to excuse so much unthinking self-exposure. At least we are now aware, it is said, of them and their plight. I am not at all sure that this has happened, that our sympathies are engaged in any true way. For as we express our sympathy, we throw off the burden; we need to do nothing as individuals, society will do it all for us.

It is not clear that the idea of the family, for example, today rests on any demanding obligation of its members to each other. Beyond this weakened notion of the family, the idea of voluntary service has declined. Charity is even dispised. Some wings of the women's movement have worked themselves into the inexcusable position of claiming that it is wrong for anyone to perform a service for nothing.

The healthy individuals are thus more and more released to go on self-improving, cultivating face-to-face relationships of slender value and lavishing their attention on bodies that are not noticeable so deserving, as society as a whole assumes all the burdens which might interrupt the appointed hour for jogging, and for lovingly mixing the bean sprouts with the cottage cheese. (But the elderly keep going, it is my observation, by heading for the meat department.) The claim that we are more aware of the disadvantaged than in the past is fraudulent and self-serving.

We may have let the handicapped and the elderly out of the closet, but by turning them out of our homes, and throwing them as wards to the bureaucracies of our societies. We ought to read carefully the recent work of men such as Michael Foucault and his disciple and colleague Jacques Donzelot on "the policing of families," their evidence that parental control but only to fall increasingly under the control of the state, and that "Family patriarchalism was destroyed only at the cost of a patriarchy of the state." We have created what is well called a therapeutic state.

To invite the state to undertake such duties is -- as the Chrysler Corporation has discovered on another context -- to invite it to exercise supervision and control. But we have placed ourselves in an even worse situation then Chrysler. We will actually pay the state to enslave us as long as it frees us from the individual obligations which we now find it too irksome to meet ourselves.

There is a further harm in it. As our soieties are compelled to fix their attention more and more on "minorities" of the disadvantaged -- actually or supposedly so -- the energy and resources that are needed to deal with general injustices in the social system are neglected and so tolerated. We may no longer sweep the elderly and the handicapped under the carpet -- if that is a fair way of describing the arrangements of the past -- but instead we set them up as totems to assure ourselves of our justice and humanity.

The fact is that we are not enabling the disadvantaged to lead "ordinary lives" by the arrangements now made for them; we are enabling them to lead extraordinary lives by conferring privileges on them as rights. I go about the city on my feet, and gaze at the ramps in the sidewalks, put there at a considerable cost. I have never once seen them used by those for whom they are intended. In Washington, the elevators provided for the handicapped at Metro stations are used by only 38 people a day.

I am then told that the handicapped cannot get there because little or no provision is made for them to get into the city on their own. So the provision of exceptional services is just pushed back one more step. If a society can afford ramps for the handicapped, it ought to be able to afford ramps and lanes for the bicyclist, so that the fit may stop using their cars. But the bicyclist is allowed no such benefits.

The majority of the fit who are the producers and earners are not encouraged to use bicycles as they are in some countries of Europe, and one has seriously to ask whether America has not given a wrong-headed priority to wheelchairs over bicycles, which might almost be taken as symbols of the argument. A civilization cannot afford to ignore those who do the work, while bestowing its largesse on minorities who necessarily contribute much less.

But according to Gov. Dreyfus, these are not only to receive the largesse, but to make no contribution to it. As the article in The Capital Times said, his proposal "carries with it a major retreat from the principle shared by all, so, too, should the cost be shared by all." But such a retreat must always be expected when particular rights are granted as privileges. Why should a wealthy couple over 65 pay no school taxes while a childless widow of 50 pays several hundred dollars? The article put the only response well: "This kind of illogic will occur when government uses irrelevant criteria such as age to levy taxes."

But that is the way in which society is drifting -- to an endless competition between the narrow claims of "minorities." The Washington Post the other day quoted the observation of Frank Mankiewicz, as he reflected on his experiences as director of George McGovern's campaign for the presidency in 1972: "We were always subject to this pressure from the cause people; we reacted to every threat from women or militants or college groups. If I had to do it over again, I'd learn when to tell them to go to hell." Wise words, too late. But it should not be too late for the rest of us.