From the very beginning of his administration, Ronald Reagan has made it clear he is deeply committed to getting rid of federal red tape and paperwork.

In his first week in office, the president created a special task force on regulatory relief and, to emphasize its importance, appointed Vice President George Bush as its chairman.

In his second week in office, Reagan froze all regulations issued during the last days of the Carter administration, pending a review to decide whether those rules were necessary. Many of them were never reissued.

Three weeks later, the president issued one of sternest executive orders ever directed to federal regulators. For the first time, they were ordered to submit all new rules to the Office of Management and Budget for review--even before the rules were publicly proposed--to make sure they were needed and that their benefits outweighed their costs. More than 2,800 rules have been reviewed by OMB since then, with 91 returned to or withdrawn by the agencies because they were found unnecessary.

In March, the White House issued its first "hit list" of rules to be reviewed and modified wherever possible to reduce costs. A total of 100 rules have been so targeted over the past year. Dozens of other rules, including many consumer-protection regulations that had become the pet peeve of many business groups, also have been targeted by individual agencies.

Altogether, these actions signaled an aggressive regulatory reform campaign and were loudly applauded by the thousands of businessmen and academics who long had complained about excessive regulation.

White House regulatory advisers have estimated that their actions to date have saved $2.8 billion to $4.8 billion in capital investment costs and an additional $1.8 billion to $2 billion in annually recurring costs.

Yet, despite this rosy report card, regulatory-reform advocates both in and out of government now admit that the administration has a long way to go to meet Reagan's campaign pledge "to get the government off the backs" of the American people.

"There is still plenty to be done, regrettably," Bush said recently.

As deregulation advocates contend:

Although more than 100 rules have been targeted for review, hundreds of others that have plagued businessmen over the years have not even caught the administration's eye. Chief among them are rules issued under the Employe Retirement Income Security Act (ERISA).

Many of the targeted rules won't be changed for months, if not years.

Dozens of laws that spawned the regulations in the first place have yet to be reviewed by the administration to see where they can be revised. Without these changes, the rules cannot be changed.

Some of Reagan's appointees have become stricter regulators instead of deregulators. The new Interstate Commerce Commission chairman, Reese Taylor, has come under the greatest attack for making it harder, rather than easier, for new companies to enter the business.

"It's proven to be more difficult to make changes than we anticipated," commented James C. Miller III in assessing the past year. Before becoming chairman of the Federal Trade Commission, Miller headed OMB's new Office of Information and Regulatory Affairs, which has been primarily responsible for the regulatory reform program.

"I thought it would go faster, but I now have learned how difficult it really is to make these changes," said Miller, who had been a frequent critic of earlier administrations for their failure to reduce red tape and paperwork.

He blames the delays primarily on legal procedures agencies face. "To change a regulation, the burden of proof is much more difficult than it is for issuing new rules. You have to prove why the old regulation is insufficient and why the new one is better." At a time when agencies are preoccupied with their budgets, the lengthy and cumbersome process has delayed many of the changes the administration had in mind, Miller said.

Yet, deregulation advocates outside the government say there are far greater problems hindering regulatory reform.

For one thing, they note, Reagan has become so preoccupied with budget and tax issues that the regulatory campaign has taken a back seat.

"They have simply pushed it down to a low level on their agenda," says Robert W. Crandall, a senior fellow at the Brookings Institution. "I see no relief in sight at this point because of the mounting deficit."

Regulatory officials also are faulted. According to Dr. Marvin H. Kosters, director of the Center for the Study of Government Regulation at the conservative think-tank American Enterprise Institute, "The administration is merely rejecting what the agencies have been doing . . . not putting forth a positive new direction for the establishment of new policies and regulatory initiatives."

This negative approach has helped to "rearm the opposition," making it increasingly unlikely that Reagan will be able to get Congress to make vitally needed changes in health and safety laws such as the Clean Air Act, the Clean Water Act and the Occupational Health and Safety Act, Crandall charges.

Administration officials are not as pessimistic about the future. "We will see a more substantial effort to draft legislation," predicted Miller.

Additionally, said Miller's successor at OMB, Christopher DeMuth, the administration will make a more concerted effort to change existing rules. "With the review mechanism in place for new rules, I think we can devote more of our time to existing regulatory programs, where the need is the greatest."

Environmental rules will continue to be prime targets, DeMuth indicated. Other priorities will be rules affecting the nation's most beleaguered industries, including autos and housing; ERISA rules, as well as dozens of other regulations that hit small businesses the hardest, and regulations that states and local governments must follow to get federal aid.

Administration officials are hopeful they can make still more changes in regulation this year. But as one conceded, it will be difficult. "We've just lived through decades where rules were constantly being added to the books . . . To extricate them, clean them off and improve them can't be done overnight."