As tensions increased along the Chinese-Vietnamese border with reports of a serious clash yesterday, Western and Asian diplomats said today that they were unclear about the cause and significance of the recent spate of hostilities, which appear to be the most serious since the 1979 war between the two communist neighbors.

In the past week, Peking and Hanoi have issued contradictory official protests and news reports accusing each other of armed provocation and of launching large-scale invasions into each other's territory, killing civilians. l

The intensity and contradiction grew today when the official New China News Agency said Chinese soldiers killed 100 heavily armed Vietnamese who had crossed the border and attacked several villages while Vietnamese artillery shelled a nearby area.

In a conflicting account, the official Vietnamese News Agency said Chinese troops invaded a North Vietnamese community, killing and wounding an undetermined number of civilians before they were repelled by Vietnamese soldiers and militia.

Although diplomatic analysts in Peking said they do not know what or who started the recent outbreak of border hostilities, some say it may stem from military moves in Cambodia, where Hanoi supports a pro-Vietnamese government that is fighting Chinese-backed guerrillas.

According to a recent report from Bangkok quoting Thai Army sources, Vietnam has shifted 1,500 troops and several Soviet-made tanks to the Cambodian frontier, close to the border with Thailand.

In a separate report quoting Western intelligence sources, China is said recently to have delivered enough weapons and uniforms to equip two battalions of noncommunist forces led by the former Cambodian premier, Son Sann.

This reported shipment along with stricter Chinese defense of its southern border is seen by some diplomats as a Peking warning to Hanoi of the costs of increasing occupation forces in Cambodia before the current dry season there ends.

Other analysts downplay the strategic importance of the recent skirmishes. Noting the absence of large troop movements on either side, they say the clashes may have erupted as a result of aggressive patrolling by nervous guards on both sides of the 90-mile border.

Whatever their analysis of the cause, most diplomatic observers say they doubt that the hostility will become as serious as the month-long war two years ago that started when Peking invaded its southern neighbor to teach it "some lessons" for its alleged border provocations.

Although Vietnam is said to have beefed up its front-line troops since 1979, diplomats say they think Hanoi would be reluctant to reopen its Chinese front while a large portion of its Army is committed to the occupation of Cambodia.

Another war would be costly for China, a nation struggling to correct a sagging economy, analysts said. Peking also would risk alienating its newly acquired friends in Southeast Asia if it struck a second time.

Despite the recent protest, analysts point out, the Chinese have avoided the strong accusations and warnings characteristic of their past exchanges with Hanoi. For example, Peking did not threaten "serious consequences," as it often does in its graver official communications.

According to Western military analysts, China has between 225,000 and 250,000 troops stationed in the two provinces facing Vietnam. Approximately the same number of Vietnamese soldiers are said to dot the hilly terrain on the other side of the border.