More than half of American men legally obligated to pay alimony or child support are in default on all or part of their payments, a new Census Bureau survey shows.

The survey, covering 1981, found that 8.4 million women had children under 21 whose fathers were absent. About 4 million of these women were supposed to receive child-support payments under court orders or legal agreements.

However, the report said, only about 1.9 million (47 percent) were receiving the full amount. One million more (25 percent) were receiving only part. The remaining 1.14 million (28 percent) received nothing.

The percentages apply only to the 4 million women who had legal orders or agreements for payments in effect at the time of the bureau survey. Of the total 8.4 women with children whose fathers are absent, only about one-third received any payment. The bureau calculated that total or partial defaults were cheating the children out of nearly $4 billion a year in support. The national total owed by the fathers under the court orders or legal agreements was $9.9 billion, but the total received was only $6.1 billion.

The average annual payment received was $2,106. It was a bit higher for whites ($2,180), slightly lower for Hispanics ($2,068) and only $1,640 for blacks.

As for alimony, the bureau said 17 million women had been divorced at some time or were currently separated, but only 782,000 were entitled to payments under court orders or legal agreements in 1981.

Of the 782,000, only 340,000 (44 percent) were receiving the full amount. Another 187,000 (24 percent) were getting partial payments, and the remaining 255,000 (33 percent) were getting nothing. Payments averaged $3,000, or about 25 percent less than in 1978 after taking inflation into account.

The bureau said that many women who had not been legally awarded child-support or alimony payments were poor and had not applied for payments because of procedural difficulties or the belief that the man could not or would not pay. Overall, of the 8.4 million women raising children whose fathers were absent, 2.566 million (30 percent) were below the official government poverty line.