It's time to get off Howard Baker's back. In fact, it's past time. During the past few weeks, we've seen a rash of stories questioning his competence as chief of staff. ''White House mismanagement," suggest Evans and Novak in The Post. ''A potted plant,'' sniggers the National Journal. ''Having trouble,'' tsks The Wall Street Journal. ''Target of mounting criticism,'' intones Business Week. And the Washingtonian asks, disingenuously, ''Who's in Charge?'' of the White House.

Just six months ago, people were saying the opposite -- that Howard Baker would ''save'' the Reagan presidency. People across the political spectrum cheered Baker's arrival. ''Finally,'' they said, ''we'll all move forward together.''

Why the sudden turnaround? Surely Howard Baker is the same man today he was then.

Partly, the media are involved. There's a premium on not being the last to pick up on a trend. Also, Washington seems to love building people up and then tearing them down. I don't suggest this is done consciously. Actually, I think it has more to do with those who want to see people cast first as heros and then as goats.

But I save my scorn for the ''insiders'' who feed the press these kinds of stories. And to judge from the reports, the chief of staff has anonymous detractors both inside the administration as well as on Capitol Hill.

Howard Baker tells anyone who will listen that his role is to carry out the policies of the president of the United States. And he likes to joke that there's a large sign on one wall of his office that reads: ''Ronald Reagan is president; you're not.'' Therein lies Howard's problem.

As with so many whisper campaigns in Washington, the object is not the person's competence but the person's policies. What is the subliminal message in the criticisms Howard is getting from Capitol Hill? ''He hasn't delivered on a tax increase.'' Well, I've got news for you: Ronald Reagan doesn't want a tax increase, and so Howard Baker hasn't delivered. Or, "Howard hasn't produced a compromise.'' But Ronald Reagan is not going to compromise his principles, and Howard Baker is not going to do that for him. Or, ''Despite Howard Baker, they have a veto strategy.'' Well, perhaps you haven't heard: Ronald Reagan is prepared to use the veto if that's what it takes to get legislation that makes sense.

What bothers the heck out of me in all this is the recurring pattern of unidentified sources. In a way, I admire those on Capitol Hill who criticize Howard Baker or anyone else on the record. I may disagree with them, but that's another matter. What really infuriates me is how ''well-placed government officials'' take potshots at people like Howard Baker and then hide behind a cloak of anonymity.

So I say to Howard's critics on the Hill: if you don't like what Howard Baker is doing, have the guts to say it publicly and let your constituents judge whether you're right or wrong. And I say to those in the executive branch: if you feel you have to take out your frustrations by leaking criticism of the president's chief of staff, you're in the wrong job.

-- James C. Miller III The writer is director of the Office of Management and Budget.