President Jaafar Nimeri has embarked on his biggest gamble in eight years in power by inviting home the man he condemned to death in absentia for leading a very nearly successful coup attempt last summer.

Sudanese now expect the return within weeks from self-imposed London exile of Sadiq Mahdi, great grandson of the Mahdi who defeated the British in 1884 at the head of his fanatical Moslem whirling dervishes.

Diplomats caution that many problems remain unsolved. They express concern about the implications of the move in this often volatile country now beset by major financial tribulations.

Symptomatic of the persistent, if unconfirmed, rumors surrounding the Saudi Arabianbacked deal struck last month, Mahdi, at 41 a former prime minister and leader of the fanatical Ansar sect of Islam, is said to be about to move straight back into a high government post.

Already there are visible signs that the surprise meeting between the president and the exile at Port Sucan July 8 is bearing fruit.

Nimeri recently released about 900 political detainees held without judicial formalities, and Monday he decreed a general amnesty for all Sudanese convicted of any offense since.

In return, Mahdi has renounced his previous insistence on reconstituting his Umma Party and presumably agreed to bring back from Ethiopia - and disarm - an undisclosed number of his Ansar followers.

Estimates of their numbers vary from a low of 2,000 to 3,000 in the view of Western observers, to 6,000 to 8,000 according to Sudanese officials and some sources put their strength as high as 17,000.

They have been a concern to Nimeri who is convinced that Ethiopia and Libya - both involved in the July 1976 coup attempt - remain tools in a Soviet plan to penetrate and dominate the Nile Valley and the Red Sea.

A further indication of Nimeri's determination to bring Mahdi back into the fold is the apparent waning of influence of his archenemy, Maj. Gen. Abdel Gassim, governor of Khartoum Province and a leading official of the country's only political party, the Sudanese Socialist Union.

As a paratroop major in Mat 1970, he dealt harshly with the Ansars who had jeered and mocked Nimeri at their stronghold on Aba Island in the middle of the Nile River. "He killed 8,000 more people than was necessary for imposing order from a government point of view," a diplomat remarked drily.

Gassim is reported to have left the Sudan for visits to the United States and Britain and is known to have rented a house in the Egyptian resort of Alexandria, presumably intending to stay away from Sudan indefinitely.

Saudi Arabia, which flew Mahdi to the Port Sudan meeting and thus acted as guarantor of his safety, acted out of a desire to have his conservative, if heretical, sect of Islam lend its considerable weight to the Nimeri government. The Saudia have chosen Sudan for enormous "breadbasket" agribusiness investments and view Nimeri as a bulwark against radical influences.

Nimeri, recently embarked on his second presidential term and known to suffer from acute high blood pressure, apparently genuinely sees his gesture as part of his grand design for national reconciliation.

Nimeri also apparently reasoned that Mahdi represented less of a danger at home where he could be watched than footloose abroad.

The hitch, however, is that Mahdi's return has shocked the Christian and Animist populations of the southern Sudan. They remember his brutally repressive policies against them when he was premier in 1966 during the worst of the violent 16year struggle against the Sudan's Moslem majority.

Nimeri, who comes from Ansar stock himself, is said to be the only northerner trusted by the south, but it is feared that Mahdi's return could sorely try the fiveyearold settlement which ended that conflict.

Moreover, diplomatic observers are also confused by conflicting reports about the loyalties of the 50,000 man armed forces. Some officers are said to favor Mahdi's return as a means of decreasing the vulnerability of Africa's biggest nation to outside attack and subversion. But others are said to fear his potentially divisive influence.

Also at play for both Nimeri and Mahdi is the assumption that the Sudan has undergone since its independence fron the AngloEgyuptian condominium in 1956 a gradual erosion of religious extremism. The growing urban population - especially the younger elite in power - is thought to be less influenced by the kind of traditional religious ties once associated with the Mahdi family.

The political dangers involved in Mahdi's return,moreover, are magnified by the Sudan's parlous financial situation. Debt repayments in terms of gross national product reached 36 per cent last month - almost twice the level which the International Monetary Fund regards as highly dangerous.

Nimeri recently flew to Riyadh for talks with Saudi Arabian officials which were expected to turn on Saudi largese to bail the Sudan out of its troubles. They were caused by a wild government spending spree following promises of vast petrodollar investments that are still lost in the bureaucratic pipeline.