AT THE RISK of perpetuating the Gilbert and Sullivan image that their grandiose title already brings to mind, we must observe that the inspector general's lot is not a happy one. The 15 watchdogs who stand guard over each of the major federal agencies must try to please -- simultaneously -- Congress, their department's higher-ups and the president, satisfy the public that they are ferreting out waste and abuse with due diligence and pursue all these duties without completely traumatizing the bureaucracies they police.

As D. C. auditor Matthew S. Watson notes on the opposite page, President Reagan has made the job harder by dismissing summarily the 15 carryovers from the Carter administration and thus highlighting the political sensitivity of the inspector general's role. Now that it's done, however, some real thought should be given to the kinds of people needed for the job. There has been talk of finding inspectors who will presumably terrorize the bureaucrats into efficiency and the recipients of benefits into lives of unblemished honesty. Would that this were all that was needed to get the job done right. The unfortunate fact is that the whole concept of the inspector general's job has been a romantic one, programmed for failure and disappointments.

The federal government runs few federal programs itself. Most are run by over 60,000 units of state and local government and tens of thousands of private organizations. How can it monitor their work? That is the challenge to the inspector general -- never mind whose appointee the lucky jobholder may be. What needs to be done is what well-run corporations do: self-checking systems must be built that promote good management up and down the line and that reduce the opportunities for fraud and abuse to minimal levels. And this must somehow be achieved without a stifling of diversity and innovation or a strangling of the system with red tape. We said minimal levels of fraud and abuse, because some fraud will get through unless you are going to post a cop in every office. A little more realism about this at the outset might avoid hysteria over the occasional headline-making fraud that accounts for a very small percentage of government waste. It would also help focus attention on the more important job of encouraging program managers to cut out the day-to-day mismanagement. It is mismanagement that, over time, adds up to billions of dollars lost -- wasted.

This is not a glamorous job, but it is one that requires understanding of how large bureaucratic systems are run and controlled. Much of the big-dollar fraud is now perpetrated by means of the very computers installed to streamline program management. And when it comes to mulcting big money, nothing now beats siphoning off federal funds temporarily into high interest bearing accounts and securities. To be able to catch this kind of abuse, the government will need to review program operations from the ground up.

Programs and audits also should be streamlined. Programs can be vastly simplified. There should be continuing audits of major programs -- not stop-and-go efforts conducted by a variety of auditors. Proably a moratorium on the stacks of incomplete audits now burdening many inspector generals' offices is required to clear the way for these improvements. Chasing down ancient abuses may be soul-satisfying, but it is not likely to pay off.

This kind of hard management job doesn't make headlines. It also takes time and an investment of money. What we are saying is that the offices that are in the news now are, when you get right down to it, in the news for the wrong reason: the story (behind the story) is that this elaborate system of safe-guards against loss of public money was built wrong in the first place and isn't working.