THE THIRD anniversary of welfare as we now know it -- the program the president and Congress did not so much create as abandon in 1996 when they shifted near-total responsibility to the states -- has occasioned the usual round of evaluations. Defenders in Congress, the White House and elsewhere proclaim it a huge success. They note not merely that the rolls are sharply down but that most of the mothers who have left the dole are working. What better evidence that the prior program, with its guarantee of support for low-income mothers whether they worked or not, was an invitation to the idleness that it supported and mistook for need?

Critics respond that the program remains untested. The test, they say, will be what happens to the children the program used to support -- about an eighth of all children in the country -- when the economy turns weak, as eventually it will; the states run out of the extra money they were given at the start of the program in return for their support; and the tougher provisions in the law take hold, including its five-year lifetime limit on welfare supported with federal funds. The decline in the caseload, which began before the law was passed, partly reflects the continuing strength of the economy and partly a new toughness on the part of the states; that much is generally agreed. But a falling caseload is not the measure of a successful policy, the critics say; if it were, the solution to the poverty problem would be to abolish aid.

Our instinct is to agree mainly with the critics. We opposed passage of the law and continue to think it was a mistake and a sellout on the part of the president. The notion of pressing welfare recipients to work is right, and plainly has helped move off the rolls some people who didn't need to be there and who otherwise would have stayed. The number of former recipients now working is impressive -- a real accomplishment. The problem is what happens to people at the other end of the spectrum, who, because of the state of the economy or their own inability, genuinely can't make it. The legislation gratuitously stripped such folk of federal protection they had previously enjoyed, and the president, who could have held out for more on their behalf, instead signed, for political reasons, a bill his own administration had earlier rightly said was insupportable. The floor beneath the poor is lower and weaker as a result, and didn't have to be.

The current evaluations only hint at all of this; no one knows for sure how the program will play out. The Urban Institute sponsored a study of women who left welfare in the period just after passage of the law. About a third had returned to the rolls by the time they were interviewed. About 60 percent of the rest were working. They were doing more or less as well as single mothers who had not previously been on welfare. That was both the good and bad news, since it meant they were having to struggle to make ends meet. Part of the problem was that they were not receiving benefits -- food stamps, Medicaid -- to which they remained entitled. Those had somehow been lost with their welfare. The study also cautioned that these women who left welfare first in a strong economy may have been the ones most capable of making it on their own. "Future groups of leavers may have different experiences, especially if they face a less favorable labor market," the study observed.

A second study, sponsored by the Center on Budget and Policy Priorities, found that incomes of low-income female-headed families had been rising just prior to welfare "reform." The average then fell in the period 1995 to 1997, despite continued economic growth; the reason was a fall-off in federal benefits. The suggestion was that vulnerable people were being stranded even in the earliest stages of welfare reform. But the data are cautionary only.

The debate about welfare reform -- on balance, did it help or hurt? -- is going to continue for years. The data will never be conclusive, in part because it is a debate about a hypothetical -- what would have happened in the absence of a change in the law? -- and in part because it is a fight about philosophy. The old welfare system perpetuated dependency; it has few defenders. The new, in which federal responsibility for child welfare has been greatly reduced, seems to us to have swung too far the other way, and to have done so unnecessarily. In tougher times, when jobs for moms are scarcer and the states are facing budget problems of their own, who will support the considerable number of children at the bottom of the economic heap? That's the unanswered question.