The General Accounting Office has ruled that the Reagan administration acted illegally when it reduced funds for the Vista program this year below a congressionally mandated floor.

The opinion could throw a monkey wrench into the administration's plans to phase out the Volunteers in Service to America by Sept. 30 on the grounds that its "limited successes do not justify the continued outlay of federal funds."

A group of congressmen succeeded in setting aside $16 million for the program in last year's budget reconciliation act, but Action, Vista's parent agency, contends that it had to cut the program by $960,000 to comply with a subsequent continuing resolution that kept the agency's fiscal 1982 funds rolling in.

That resolution, enacted last December, called for an overall 4 percent cut in Action's appropriations but stipulated that no program or project could be cut by more than 6 percent. To maintain Vista's funding at $16 million, Action argued, would mean cutbacks of more than 6 percent in other programs.

But the GAO, in an opinion issued Wednesday and signed by general counsel Milton J. Socolar, said the wording in the earlier act was clear: $16 million was appropriated specifically for Vista and only Vista, and could not be spent on anything else.

The opinion acknowledged that rearranging the budget six weeks before the end of the fiscal year could pose some problems for Action, but added: "The possible consequences of adherence to our decision cannot . . . alter our legal conclusion, which is required by the clear wording of the relevant statutes."

Action director Thomas Pauken, who called the GAO letter a "difference of opinion among attorneys," expressed concern yesterday that putting money back into Vista might result in cutbacks in the Older American Volunteer Programs--which include the foster grandparents, senior companion and retired senior volunteer programs, all staunchly supported both in Congress and in the Reagan administration.

Pauken's critics have some other ideas. They would like to see Action cut its administrative staff in Washington or scrap a new teen-age volunteer program.

A staff aide to the House subcommittee on select education, whose chairman, Rep. Austin J. Murphy (D-Pa.), was among seven members of Congress who recently requested a larger GAO investigation into practices at Action, said the agency's administrative budget has risen 9 percent at a time when programs have been cut and career employes have been laid off.

Pauken disputes that, and says the increase in administrative spending this year was caused, in part, by the one-time expense of the RIF. The fiscal 1983 budget, he said, will reflect lower administrative costs.

Meanwhile, Action announced last week in the Federal Register that it will have up to $900,000 available for fiscal 1982 grants for the teen-age volunteer program, called Young Volunteers in Action. Pauken's critics point out that $900,000 would almost cover the missing Vista funds.

But Pauken clearly would rather not sacrifice YVA, which is patterned on the highly successful senior citizen volunteer programs. "Vista is a program of the '60s," he said. "Why is the Reagan administration not entitled to its own initiatives?"