Alice M. Rivlin, founding head of the Congressional Budget Office and sometime critic of the economic policies of four presidents, including President Reagan, has said she does not intend to seek reappointment when her term ends Jan. 31.

She told CBO staff members of her decision Friday, and in the last few months has given congressional leaders the same message, an aide siad.

"We're going to ask her to stay on," House Budget Committee Chairman James R. Jones (D-Okla.) said yesterday. But "We are going to begin to consider who would replace her," he added.

If Rivlin is determined to leave the influential office she organized seven years ago, he said, "this is probably as good a time as any."

Still, Jones said, "it's not going to be easy to replace her," and " she has indicated to me she is certainly not going to leave the budget process in a time of need."

High among possible replacements being considered is Rudolph Penner, an economist and resident scholar with the conservatively oriented American Enterprise Institute. He served as chief economist for the Office of Management and Budget under President Ford.

Also recently considered was Harvard economist Martin Feldstein, but that possibility ended last month, when Reagan named him chairman of the Council of Economic Advisers.

Rivlin came under heavy fire from some conservative Republicans this year and last when she forecast a far greater deficit than that projected by the president and her agency indicated that large tax increases would be needed to cut the deficit.

The criticism largely has abated as her forecasts proved correct, and Congress passed a $98.3 billion tax increase at Reagan's urging. She nonetheless would have to win reappointment from a divided Congress.

"If you look at the last few years," Jones said, "Alice has at various times been either on the 'good guy' or 'bad guy' list" of almost every side in Congress. That just points out how independent and nonpartisan she really is, he said.

Charles L. Schultze, chairman of the Council of Economic Advisers under President Carter, whose policies Rivlin also criticized at times in her plain, no- nonsense style, said:

"In the face of some very difficult problems, she has established a strong, highly competent and nonpartisan organization which has become indispensable to the whole congressional budget processs. "

Rivlin, 51, was little known publicly but was respected as an economist at the liberal Brookings Institution when Congress passed the Import budget reforms that created the CBO in 1974.

Its mission was to help Congress start setting firm annual budget ceilings. She took it over in February, 1975, and almost from the beginning was accused by some Capitol Hill critics of trying to set national policy instead of just reporting facts.

She replied that her agency "does not make recommendations on policy. It does show what it thinks the effects of alternative policies would be."

Rivlin was out of the city yesterday, but James L. Blum, CBO's assistant director for budget analysis, said, "It's true that she's not seeking another term. This has been an excellent year for her. The analyses we've done have been on the mark and well-received.

"It's her feeling that two terms -- eight years -- are enough . . ."