WE CAN'T SAY we were bowled over by the announcement that Secretary of Health, Education and Welfare Joseph A. Califano Jr. is prepared to cut off federal funds from schools that persistently violate the laws against discrimination based on race or sex. We would have been stunned if he had said anything else. After all, cutting off federal funds is what the law directs him to do when all else fails. And we have assumed all along that Mr. Califano will do what the law tells him to do.

So what is important about Mr. Califano's announcement is not so much his threat to use the ultimate weapon at his command. What is significant is his profesed intention to step up the pace at which HEW is supervising the elimination of discrimination from educational institutions.In seeking to do that, Mr. Califano may need to do much more than just reopen some old cases, pursue compliance reports based on exisiting regulations, and threaten fund cut-offs. We have the distinct impression that in some areas of fighting discrimination HEW has contributed more to the problem than to the solution. Some of its regulation have board on th ludicrous and thus have fostered more resistance that complaince. And sometimes its approach to particular problems has smaked more of a mailed fist than of reasoned persuasion.

In light of HEW's record in the past few years, Mr. Califano may have felt it important to remind school officials he has the power to cut off federal aids. But having it and using it are two different things. The schools to reach that goal, not to punish them for falling short. While the existence of the power to cut off federal funds seems necessary to convince some school officials the government means business, its exercise is to be avoided if at all possible. Mr. Califano would do well to introduce into HEW's antidiscriminationality, reasonableness, persuasion and compromise - that often can get the job done while eliminating the need to use force in doing it. By the time you have to cut off funds, you have lost of this exercise.