Housing Secretary Jack Kemp, an open mouth in a closed-mouth administration, stirred real animosity in the upper reaches of the Bush hierarchy only when he challenged the decision by New York Republicans to put pragmatism first in picking their nominee for governor.

Kemp encountered a little flak from the administration by publicly differing from President Bush's stands on the Baltics and the budget. But that was nothing compared to expressions of resentment over his criticism of political unknown Pierre Rinfret to oppose the formidable Gov. Mario Cuomo's bid for a third term.

Kemp's complaint that Rinfret had been trashing the Reagan economic heritage won him no sympathy from the Republican high command. ''Jack's talking about ideology, and a campaign against Cuomo should have nothing to do with ideology,'' a Republican politician high in the Bush counsels explained to us. ''We're just trying to cut down Cuomo a little and stop the Democrats from taking over the state Senate.''

To say that ideology has nothing to do with running against the leader of the Democratic left is a reversion to form for the GOP. For most of the past half century, Democrats have been more ideological (and successful) than Republicans. The GOP's step away from pragmatism under first Barry Goldwater and later Ronald Reagan looks aberrational. After the Reagan Revolution brought them to parity with the Democrats, Republicans have turned to less ideological candidates nationwide.

That was true in New York, where volunteers to run into the Cuomo juggernaut were in understandably short supply. Willing to make the plunge was New York University Dean Herbert London, the choice of the Conservative Party. But state Sen. Roy Goodman, a Manhattan liberal emerging as kingmaker, vetoed London and unveiled Rinfret.

Rinfret is dynamic, charming and a self-made millionaire who built his economic consulting firm from scratch. But since advising the Nixon White House two decades ago, his ties with the GOP have so atrophied that records show he has not voted in party primaries or even registered as a Republican. Party loyalty, as with economic philosophy, always has been eclectic for Rinfret.

So why pick him? The answer that he brings $500,000 of his own money to the game fails on two counts: 1) That's chicken feed in Empire State politics; 2) Rinfret has downsized the number to $100,000.

The real reason is that Goodman is described by a state party leader as ''maniacal'' in fear that the horde of abortion rights female Democratic candidates will use the abortion issue to defeat antiabortion male Republican incumbents and win control of the state Senate. Pro-choice Rinfret passed Goodman's litmus test; pro-life London failed it.

Kemp, who no longer votes in New York since his retirement from Congress, could not keep quiet about it. He called the state party's official stand for abortion rights ''a shame'' and ''sad'' and allowed that Rinfret's attacks on the Reagan economic record made it easier for him to support London.

No greater heresy could have been committed by a Bush Cabinet member. ''If he had any guts,'' New York political operative Rich Bond, a longtime Bush insider, told an associate, ''he'd have run for governor himself'' -- a reference to Kemp's refusal in 1982 to seek the then-open governorship, which was won by Cuomo.

Ed Rogers, newly installed as White House political operative, telephoned Rinfret to assure him of the president's support. Kemp was urged by a presidential aide to follow Bush's path. Faithful Kempites, old comrades-in-arms from the House and doughty veterans of his failed presidential campaign, sadly shook their heads that their leader had gone off the tracks. Among Republican politicians in Washington that we contacted, only New Right activist Paul Weyrich approved Kemp's course.

Lawrence Kudlow, a former Reagan administration official and current New York GOP vice chairman, has been trying to arrange a Rinfret-Kemp peace parley (while talking Rinfret into trashing an insulting personal letter to Kemp). The real Republican hope, however, is that Kemp will just shut up.

''I can't figure out what Jack is up to,'' one Bush aide told us, adding, in a more sinister tone, that ''it sounds like he is pursuing a personal agenda.'' The difficulty of understanding anybody guided by ideology suggests a party galloping away from the Reagan victory formula and back to the comforts of pragmatism.