Representatives of 38 Islamic countries opened talks here today in an effort to work out a united stance against the Soviet invasion of Afghanistan, but demands by pro-Soviet and anti-Israeli delegates immediately threatened to sidetrack the conference.

As the talks opened, six rival Afghan rebel groups announced that they have combined into the Islamic Alliance for the Liberation of Afghanistan. The move clearly was timed to present the conference with a broad-based nationalist organization to receive financial aid for the fight against the Soviets.

[Meanwhile, Soviet Foreign Minister Andrel Gromyko arrived in Syria for talks aimed at cementing ties with that increasingly isolated Arab country. Details on Page A14.]

Pakistani President Mohammed 2ia ul-Haq, in the only public session of the conference, called in his opening address for harsh action against the Soviets, but he did not specify what steps the conference should take.

Representatives of Moslem countries with close ties to the Soviet Union quickly began moving to water down any resolutions of condemnation, according to sources present at the closed-door sessions that followed Zia's remarks.

Other Arab states reportedly were attempting to broaden the conference's agenda to include attacks on the Egyptian-Israeli peace treaty. In addition, Iran has said the conference should condemn U.S. economic pressures directed toward freeing the American hostages being held in Tehran. p

Although Saudi Arabia reportedly is asking the Moslem states to break diplomatic ties with Moscow, knowledgeable Pakistani sources said tonight that it appeared the moves against the Soviet Union would take a much milder form.

These, they said, could include:

Pledges of aid to the 600,000 Afghan refugees who have flocked into Pakistan since a pro-Marxist government seized power in a bloody coup in April 1978.

Condemnation of the Soviet Union's military intervention in Afghanistan, although it is likely to follow the form of the resolution passed by an overwhelming vote by the United Nations General Assembly and not mention the Soviets by name.

Along with this, secret agreements could be made to supply aid to the Afghan rebel forces.

The possible call for a boycott of the Summer Olympic Games in Moscow, although each nation would be free to send its athletes to compete if it wished.

In his opening remarks, Zia said it would not be enough to "demonstrate by words alone our grave concern at the flagrant violation of the sovereignty and national independence of brotherly Afghanistan, whose proud people have a glorious heritage of Islamic values, traditions and culture."

Some conference members, however, have close ties to the Soviet Union and even the conference secretary, Gen. Habib Chatti, was at pains to point out that the Soviets often had lent their support on "matters of vital and crucial importance to us."

Among those members with close ties to Moscow are the Palestine Liberation Organization, which gets most of its arms from the Soviets, and Syria, which is not attending this meeting. Egypt is not represented among the delegates, the majority of whom are foreign ministers, nor is Afghanistan.

Both Chatti and Agha Shahi, Zia's foreign policy adviser and the conference chairman, made it clear that any actions the delegates would take were neither anti-Soviet nor pro-American but instead aimed specifically at Moscow's intervention in Afghanistan.

Agha Shahi called the Soviet move into Afghanistan "the most recent manifestation of the bid for hegemony" that has made the Islamic states "the focus of global rivalry" between the United States and the Soviet Union.

It remained unclear tonight, after an evening session that lasted far past the scheduled adjournment hour, whether these efforts to narrow the scope of any anti-Soviet action to Afghanistan alone would allow Moscow's Moslem friends to support strong resolutions here.

The conference appears likely in any case to reinforce its neutral position between Washington and Moscow, emphasizing again that the world's 800 million Moslems are not aligned against the Soviet Union.

That, observers here said, could be turned into a diplomatic voctory by Moscow, especially if there are enough points in the final resolution that clearly work against U.S. interests.

Meanwhile, a passing reference in Zia's opening speech threatened to throw a monkey wrench into recent efforts to narrow the wide gap in understanding between the neighboring states of India and Pakistan, which have fought three wars since they were carved from a British colony and given independence 32 years ago.

In one line of his speech, Zia mentioned that the largely Moslem population of Kashmir -- an Indian state -- along with Palestinians are "yearning for a restoration of their rights."

The Indian ambassador here, K. S. Bajpai, invited with other diplomats to the opening session, walked out when he read that reference in the text of Zia's speech and filed a quick protest with the Pakistani Foreign Office.

The incident took place at an untimely moment. The Indian foreign secretary, R. D. Sathe, is due here next week for talks on defusing the differences between the two countries about the Soviet intervention in Kabul. dIndia has opposed any renewed arms sales to Pakistan and insisted that the Soviets will not move beyond the Afghan borders.

The announcement today of a united Afghan rebel front climaxes more than three weeks of work to get the rival groups together. They tried in August, with the promise of $1 million from Saudia Arabia if they united, but the alliance lasted only a short time.

There were indications of splits between the different groups at today's press conference as representatives of each supplied slightly different answers to questions. But, one diplomat here said, getting them all together in one room represented a great achievement.

It is still questionable, however, how strong the links are between the six groups that joined in the Islamic Alliance based in the Pakistani border city of Peshawar and the bulk of the rebel Moslem tribesmen fighting in Afghanistan.