Twice President Reagan has recalled on nationwide television that he dropped 350,000 people from the California welfare rolls without pain to the truly needy and deserving.

"In California," he said Thursday night, "when I was governor and embarked upon welfare reform, there were screams from those who claimed that we intended to victimize the needy. But in a little over three years we saved the taxpayer some $2 billion at the same time we were able to increase the grants for the deserving and truly needy by an average of more than 40 percent. It was the first cost-of-living increase they have received in 13 years."

On March 6, Reagan said, "We never had a single case of anyone appearing and saying 'I am destitute. I've been cut off welfare.' "

But some California scholars and lawyers recall it differently.

In an old controversy with new, national implications, welfare experts say the decline in California relief rolls in the early 1970s resulted mostly from a fortuitous upswing in the state's economy.

Experts here still express admiration for Reagan's modest success at tightening California's welfare procedures, but they dispute some of his claims about how this tightening was carried out.

Ralph Abascal, now general counsel for California Rural Legal Assistance, said he filed 40 lawsuits in the early 1970s on behalf of welfare recipients about to become destitute because of Reagan's program.

Abascal, asked for his reaction to Reagan's remarks, said, "In one word? Disgust. Disgust at the Reagan people and their manipulative use of statistics and frustration with the press for their inability to cut through that."

However, Robert Carleson, Reagan's welfare director in Sacramento, and chief welfare adviser in Washington, said it is actually the president's critics who are manipulating figures.

"Even to this day they ignore the fact that not only did the rolls drop by about 300,000, but the state budget analyst and many of Reagan's critics predicted welfare rolls would increase by 500,000 during that same period," said Carleson, special assistant to the president for policy development.

Reagan has boasted for years of a drop in California welfare rolls. The most detailed critical analysis of his claim occurs in a little-noticed 1976 report by welfare experts Frank Levy, now a senior research associate at the Urban Institute in Washington, and Michael Wiseman, now an associate professor of ecnomics at the University of California at Berkeley.

They concluded that, although Aid to Families with Dependent Children dropped 278,000 people from its rolls from January, 1971, to August, 1974, only 25 percent of that drop could be credited to Reagan's welfare reforms.

"The California recession peaked in 1971 and then declined through 1973 before becoming severe again in the second half of 1974," the report said. It noted that this period matched the time of sharp decline in welfare rolls.

Reagan also benefited, the report said, from a steady decline in the birth rate caused by greater use of birth control and a new state law easing abortions. "Welfare families, like families elsewhere, were having fewer children," the report said.

In the decade ending 1976, the report said, the average number of children with one parent or with a disabled father declined 26 percent, from 2.85 to 2.1 children. The number of children in welfare families with the principal wage-earner unemployed dropped from 4.01 to 2.57, or 36 percent.

In a telephone interview, Carleson said Levy and Wiseman missed the point. He acknowledged that the improving economy helped reduce the rolls, "but that was our goal, to make sure the rolls did fluctuate with the economy."

Before Reagan toughened the income standards for welfare clients and tightened checking procedures, such as requiring a monthly income report before recipients received their money, the welfare rolls increased whether the economy was good or bad, Carleson said. "Everybody got real quiet when they went up in good times," he said.

"The decline in family size should also be attributed to the reforms," Carleson said. Once the reform program began, "they had to prove they really had three kids."

Wiseman called "preposterous" the claim that Reagan's checking procedures brought the recorded decline in welfare family size. "In all my years," Wiseman said, "I have never run into a case of the number of dependents being falsely reported."

Carleson, who spent much of the 1970s advising other states how to reform their welfare systems, said he noticed welfare rolls stabilizing in states that adopted tougher checking procedures, and continuing to climb in states, such as Massachusetts, that did not.