President Reagan took the offensive yesterday against charges that his budget is unfair to the poor in cutting social programs while increasing spending for defense.

He denied the social safety net is tattered and said his administration was doing more for the truly needy than any before it.

"In the history of mankind there has never been a people who have strived harder or done more than we Americans to help all who are truly in need," the president told listeners to his weekly radio broadcast from the White House. "This administration is committed to carry on that tradition."

The president said that ill-informed criticism of his budget proposals had come about because of "long-overdue attempts to target benefits to the truly needy and to reduce benefits for those who should be able to manage for themselves."

Polls for more than a year have shown Reagan increasingly to be perceived as uncaring about the poor and the problems of minorities and women, both for the trends in his budget and for his opposition to some federal programs directed at the poor, such as Legal Services.

In addition, record unemployment rates and Reagan's reluctance to agree to a federal jobs program until last week added to the image that the president is not concerned with victims of the recession.

Last year Reagan's deputy chief of staff, Michael K. Deaver, said this "fairness" issue was damaging the president politically.

"What about their charge that we're slashing spending on social programs to spend more on defense?" Reagan asked his listeners yesterday. "Well, it's true that we're requesting $1.6 trillion in defense spending over the next five years. But I'll bet you haven't heard that during this same period spending budgeted for entitlements programs will be over $2 trillion, [which is] more than defense."

Later Reagan said: "What we propose to spend on defense is a much smaller part of the federal budget and our total economy than was being spent 10, 20, 30 years ago. Yet the threat to America's freedom is greater . . . so let me repeat, far from trying to destroy what is best in our system of humane, free government, we're doing everything we can to save it."

In the Democratic response to the speech, Rep. Al Swift (D-Wash.) said Reagan's approach to relieving the nation's 10.4 percent unemployment rate "isn't fair" because the White House believes unemployment is "unfortunate but necessary."

"It simply isn't fair to those who cannot find work, those who can only work part time and those whose jobs hang by a slender economic thread," Swift said. "The wealthy, the corporations, big business were not asked to accept such sacrifice when the tax cut was proposed."

Reagan touched on the unemployment issue in his speech, saying unemployment compensation plus welfare and medical, food and housing assistance for the poor would be 24 percent higher under his fiscal 1984 budget than it was in fiscal 1981. One reason for the increase is the rise in unemployment, however.

"In our fiscal year 1984 budget, we've proposed $93 billion in assistance for the needy and unemployed," Reagan said. "Twenty-three years ago, the federal governemnt wasn't spending $93 billion on its entire budget."

Further defending his budget for social programs, Reagan highlighted administration efforts in nutrition assistance. He said "doom and gloom criers" had charged that he was "increasing hunger." But the government now subsidizes 95 million meals a day, he said, more people are food stamp recipients and "average benefits per person have grown at a rate faster than food price inflation."

Reagan then said that his economic recovery program would help the entire nation: "The most unfair situation was the one our people were trapped in before, when record inflation, taxes and interest rates were slamming shut the gates of prosperity on every American family."