Productivity in the nation's private businesses, other than farms, fell at a 4.1 percent annual rate in the second quarter of 1974, the Labor Department reported yesterday.

The recession caused output of the businesses to plummet at a 12.8 percent rate, the largest decline since the department began keeping these statistics in 1947. The number of hours worked also fell at a 9.1 percent rate, reflecting both shorter workweeks and massive layoffs.

Meanwhile, the compensation paid per hour worked continued to rise at a faster rate, the department said. Last year the combination of wages and fringe benefits paid by the businesses rose 9 percent. In the first quarter of 1980, compensation jumped at a 10.7 percent annual rate and continued upward at a 10.8 percent rate in the most recent quarter.

The fall in productivity and the increase in compensation together meant that until labor costs, which often move in line with inflation, climbed at a 15.6 percent annual rate, up sharply from the first quarter rate of 12 percent and half again as large as the 10.2 percent rate for all of 1979.

If the agricultural sector is included, the overall decline in productivity is somewhat less, dropping at a 3.1 percent annual rate.

In manufacturing, productivity fell at a 3.2 percent annual rate. Businesses suffered a whopping drop in output -- at a 20.8 percent rate of decline. But by laying off workers and shortening the workweeks of their remaining employes, manufacturers cut hours worked at an 18.2 percent rate, nearly offsetting the fall in production.

The productivity decline was even smaller in the durable goods manufacturing sector, the one hit hardest by the recession since it includes the auto industry. Among durable goods makers, output plunged at a 24.4 percent rate, but layoffs and shorter workweeks reduced hours worked at a 23.3 percent rate -- leaving only a 1.4 percent drop in productivity.