Builders spent 1.1 percent more on construction labor and materials in November than October, an annual rate of $322.2 billion that is 20.7 percent ahead of a year earlier, the government said yesterday.

The increase reported by the Commerce Department Census Bureau compares to very small changes in September and October. October's increase was less than a tenth of a percent.

Private construction gained 1.3 percent in November although single family houses were down 0.3 percent. Public construction spending was unchanged, despite a large 5.3 percent increase for highway and street projects.

Private office building construction, favored by tax incentives, continued to boom with a 5.6 percent increase, despite the view of analysts that most sections of the country already have too much office capacity. Spending on office buildings has rocketed up 39.8 percent in a year.

The monthly survey of construction spending reflects reports from most large construction projects in the nation and a sampling of the smaller ones to track month-by-month progress.

Without seasonal adjustment, there was $287.3 billion in spending on construction labor and materials in the first 11 months of 1984, 19.2 percent ahead of the same period last year.