For the first time in years, China has reported a significant failure in its birth control program, indicating that new economic policies may actually be worsening the population crisis in the world's most populous nation.

An official radio broadcast from Kwangtung, the fifth-largest province in the country, said more than 100,000 births in excess of an official provincial plan were extended this year. Analysts here who have followed China's regular reports of birth control successes in the last decade said this was the first time the Chinese have reported a significant birth rate increase anywhere, indicating the seriousness of the failure.

The resurgence of births in Kwang-tung closely follows implementation of new, nationwide rural policies promising more income to peasant families who do more work. This also provides an unintended incentive to produce bigger families. The policy may increase production of grain, but "the peasant will also say to himself, 'my family can really get ahead if I produce a lot of strong children," said one analyst.

The Chinese government is the first in history to try to administer nearly one billion people. If it does not reconcile its drive to increase food production with its need to reduce births, living standards are expected to stagnate or decline and moderinization plans to be severely crippled.

A number of provincial and national conferences on the population problem have been held this year, including a national conference on population theory that just ended in Peking. The Kwangtung broadcast and some other articles have blamed lingering rural tradition for the failure to cut birth rates more quickly, but several recent articles criticizing officials for interfering too much in peasant life may also have contributed to falures such as Kwangtung's.

The articles chided officials for interfering with peasant decisions on what crops to plant. But they may also have discouraged local leaders from disciplining people who refused to practice birth control.In the past, families sometimes found free medical and other services might be denied to their third or fourth children, a harsh measure that contrasts with the new, relaxed atmosphere the post-Mao Chinese leadership is trying to encourage.

The Sunday Broadcast from Kwangtung, monitored here, hinted at a conflict between birth control and other policies when it complained that "some places have set implementing the rural economic policies against implementing the policy of planned parenthood, and have relaxed leadership over this work."

"The number of pregnant women and births not included in the population plan have increased, compared with the past. This will certainly cause a rise in the population growth rate." the broadcast said. "The increase in population in excess of the plan will cause a number of problems in food, clothing, housing and transport, thus adversely affecting any rise in the people's living standard and delaying the four modernizations."

Kwangtung, a large province along the South China coast with its capital at Canton, has about 50 million people, most of them peasants. The broadcast said that up until this year's reversal, te population growth rate had dropped from 29.4 per thousand in 1965 to 12.61 per thousand in 1977.

In February, Communist Party Chairman Hua Kuo-feng announced a national goal of reducing the growth rate to less than 10 per thousand within three years. This seemed possible in some urban areas. Shanghai, the most populous city in China, has the world's lowest reported birthrate, only 6 per thousand. But foreign population analysts guessed that the growth rates were much higher in many rural areas, where the Chinese have been reluctant to release figures. The Chinese have not conducted an official census since the early 1950s and foreign experts estimate the total population at from 900 million to more than one billion.

The official press has indicated for more than a year that the government is requiring fewer high school graduates to leave urban areas and work in the countryside, a policy change that may make it much more difficult even for cities to control population growth. Perhaps feeling the new population pressures, Peking has so far turned a deaf ear to complaints from Hong Kong about the approximately 100,000 legal and illegal immigrants, many from Kwangtung, who have flooded this British-run city because of relaxed Chinese border controls this year.

The Kwangtung broadcast blamed much of the birth rate increase on some leaders . . . who set a bad example themselves. For instance, five tof the top leaders of a county party committee in Meihsien Prefecture have each produced five or more children since 1973: Two have produced a sixth child and one a seventh. The growth rate in that county was expected to jump from 13.46 per thousand in 1976 to 19 per thousand this year, the broadcast said.

The broadcast said in one commune where a leading official just had his seventh child, the birth rate was 26.94 per thousand last year and would climb to 30 per thousand this year.

Since the late 1950s, the Communist Party has experimented with a number of different systems for rewarding peasants who work their fields colllectively. In the last few years of the life of Chairman Mao Tse-tung, his egalitarian policies were particularly influential. Local officials often encouraged systems that distributed food and cash to peasant families based roughly on what they needed, with little extra income for extra work. This year, however, rewards for extra efforts are being encouraged and peasants are allowed to raise pigs and cultivate private gardens in their free time.

No provisions have been made yet for government-guaranteed state support for retired peasants, however. Sons still are generally expected to support their elderly parents, a system that encourages families with only daughters to keep having more children. CAPTION: Map, no caption, By Dave Cook - The Washington Post