The General Accounting Office sharply criticized the Labor Department's controversial summer youth program yesterday charging that youngsters often are paid for doing little or no work or for simply playing basketball all day.

In a report to Congress, the agency said its survey of four urban-area programs and three rural efforts showed anywhere from 20 percent to 75 percent of the youths enrolled were not involved in the sort of "meaningful" job experience they are supposted to be obtaining.

It recommended that, before considering any increase in the $755 million program, Congress force the Labor Department to crack down on local sponsors. It said the purpose of the summer program "is defeated when youths are paid for doing little or no work or for playing games."

The document was the latest in a series of disparaging reports that GAO has prepared on the summer youth program in the past few years. A draft was made public yesterday by Sen. Lawton Chiles (D-Fla.), one of the program's most vocal critics.

The program is an outgrowth of the old Neighborhood Youth Corps of the Great Society days, which was criticized for providing little more than income maintenance. The present summer youth program is supposed to give youngsters experience to prepare them for jobs later on.

As in previous years, the GAO was particularly critical of the Labor Department's administration of the program, charging that its rules and regulations were not worded tightly enough and that regional offices did not monitor local efforts closely enough.

The GAO said inspection visits by its investigators often uncovered inadequate supervision and little thought or planning in the "jobs" parceled out to disadvantaged youth. It said many of the youngsters showed up only for part of the day and then went home.

Chiles is expected to pursue the issue today at a hearing of a Senate Appropriations subcommittee. The Carter administration has asked Congress to scale the youth program back in fiscal 1980 by raising the age for admission to 15 years from the current 14.