The Democratic-controlled House, with help from Republican defectors, handed the Reagan administration major setbacks yesterday by defeating efforts to scale down a new farm bill's dairy and sugar price supports.

The administration had joined with consumer advocates in a lobbying effort to revamp both the dairy and sugar provisions of the Agriculture Committee's bill, but each was defeated -- by surprisingly large margins -- on roll-call votes.

A proposal by Rep. James D. Olin (D-Va.) and House Minority Leader Robert H. Michel (R-Ill.) to reduce milk surpluses and cut federal spending on the dairy program was defeated 244 to 166. The committee's approach, which would pay farmers to cut production, remained intact.

An administration-backed amendment by Reps. Thomas J. Downey (D-N.Y.) and Willis D. Gradison Jr. (R-Ohio) to modestly scale down the committee's sugar support plan was defeated by a wider margin, 263 to 142.

On the dairy amendment, 74 Republicans voted against Olin and Michel. On sugar, 103 Republicans voted against the administration.

House Agriculture Committee Chairman E (Kika) de la Garza (D-Tex.) said that yesterday's votes indicated growing concern among House members about the deteriorating farm economy and a general realization that "the administration program gives the farmer the back of the hand."

And Rep. Edward R. Madigan (R-Ill.), ranking Republican on the committee, said the rejection of administration officials' positions on dairy and sugar "may demonstrate to them that the problem in the farm community is much more severe than they thought it to be."

Agriculture Secretary John R. Block called the votes "regrettable." He said, "To perpetuate current policies that have clearly failed is a tragedy. American farmers are the biggest losers in today's House action."

In both cases, close votes had been expected. The White House had promised a strong lobbying effort and consumer groups joined with congressional critics to argue that the committee's bill would wrongfully add to retail food costs.

But one Farm Belt legislator after another took the floor yesterday to plead for the committee's hold-the-line bill, on the ground that the farm economy, in its worst throes since the Great Depression, could not stand the belt-tightening sought by the administration.

Not even impassioned oratory by members such as Rep. Barney Frank (D-Mass.), who complained about programs subsidizing farmers not to produce and others subsidizing farmers to overproduce, made a dent.

Frank at one point called the committee's dairy "diversion" plan, which would pay dairy farmers who agree to reduce production, "guild socialism" that would lead to "subsidizing everything short of tofu." The federally sponsored butter surplus, he said, was big enough "to slather Wyoming into complete slipperiness."

The committee bill would slightly increase the federal milk support level next year but allow it to climb almost $2 over the five-year life of the bill if farmers bring surplus production under control. The "diversion" would be financed by assessing a fee on all milk production and would allow the government to buy entire herds from farmers who want to leave the business.

Olin and Michel proposed a simpler approach -- cutting price support levels until farmers found it unprofitable to overproduce. Opponents agreed that surpluses had to be curbed, but that the sharp support cuts would drive farmers out of business.

The sugar provisions, extending a complex system of quotas and tariffs to keep lower-cost foreign sugar off the U.S. market, would continue price supports at 18 cents a pound. Downey and Gradison proposed a yearly 1-cent reduction.

The House is to resume deliberations Tuesday.