An initiative by the Chilean government to permit return of some of the country's thousands of political exiles has been curtailed by President Augusto Pinochet in a recent toughening of government rule, according to human rights leaders and diplomats here.

Pinochet raised expectations of a major political liberalization of his nine-year-old rule late last year by forming a high-level commission to study the return of exiles, one of the most debated human rights issues here in recent years. Although no official figures exist, Chile is believed to have one of the highest proportions of exiled citizens in the world.

Following a debate among the commission members and government officials, however, the initiative on exiles has been scaled back significantly, human rights groups and other sources said. Government officials have authorized the return of only 309 persons since late December, and no political leaders have been included on the list.

In addition, Pinochet dissolved the special exiles commission shortly after it delivered its report in December, and the group's recommendations were kept secret despite previous government indications that they would be released. Informed sources said the commission had recommended an end to entry restrictions on more than 500 exiles, including several dozen leaders of centrist political parties--a plan firmly rejected by Pinochet.

The apparent reversal has disappointed government supporters who have pressed recently for an easing of repression and led to opposition charges that the widely publicized program was only a gesture to improve Chile's international image.

"It was an absolute fraud that demoralized a lot of people, because the illusion they created was of a mass return of exiles with only a few exceptions," said Jaime Castillo, a Christian Democratic Party leader and president of the Chilean Human Rights Commission.

Castillo, who has become one of the best known Chilean exiles since his expulsion from the country in 1981, said in an interview in Caracas, Venezuela, that Pinochet appeared to have curtailed the exile plan in reaction to a series of antigovernment demonstrations and a renewed censure of his rule by the U.N. General Assembly in December.

"He seemed to react and say, 'If these people want to bother me, I'll drop the whole matter,' " said Castillo, who is one of several leaders whom human rights groups expected to be allowed entry following private indications from official sources.

Both Chile's Catholic Church-based human rights organization and the more politically based commission headed by Castillo now say the slow movement on the exile issue is part of a broader tightening of Pinochet's military rule in recent months in response to an upsurge in political opposition and public unrest. In its annual report for 1982, the commission reported that government repression had reached its highest level since 1977, when the military's campaign against supporters of the late Socialist president Salvador Allende and other government opponents was still in progress.

The number of political arrests recorded by the group last year nearly doubled from 1981, to 1,789, and cases of alleged torture increased from 61 to 100. Several political demonstrations were violently broken up by government security forces late last year. This month, Pinochet engaged in his first direct confrontation with Catholic Church authorities in six years through the expulsion of three foreign priests.

Reagan administration officials, who have sought to improve U.S. ties with the Pinochet government and who argue that human rights conditions have improved since 1977, concluded in the 1982 human rights report to Congress that "the pace of improvement has slowed in the past two years."

U.S. officials also say the administration has all but abandoned plans to certify Chile to Congress this year as having improved on specific human rights issues, a condition for the resumption of U.S. military aid suspended during the Carter administration.

Although Pinochet's government has been criticized on a range of human rights issues, the status of exiles has recently become the most politically sensitive theme.

Chile's ambassador to the United Nations said recently that political exiles numbered about 11,000--or about one of every 1,000 Chileans. The government has consistently refused to release lists of those prohibited from entering the country, however, and independent estimates range from the 10,000 persons and 20,000 family members cited by U.S. officials to 200,000 calculated by the Chilean Human Rights Commission.

Thousands of those living abroad left the country or sought asylum in embassies as active supporters of the Allende government following the violent military coup that overthrew him in 1973. But many others were later expelled or prohibited from reentering the country after trips abroad, including a number of leaders of Chile's centrist political parties and labor movements.

The latest such action came early last December--even as Pinochet's commission was meeting to consider the exiles issue. Opposition labor leaders Hector Cuevas and Manuel Bustos and a conservative farm organization leader, Carlos Podlech, were expelled for helping to organize demonstrations prohibited by the government's ban on all political activity.

In a number of cases, citizens have learned of their exile only upon arriving at Chilean immigration stations, where inspectors check the names of every person entering the country. The inspectors use computer terminals or in printed lists as thick as a metropolitan telephone book.

For prominent government opponents, there has been no hint of flexibility. Last Friday, for example, authorities denied a request by Allende's exiled former foreign minister, Clodomiro Almeyda, to enter the country briefly for the funeral of his mother.

Government officials, who refused to be interviewed on the exile issue, have said that most of the exiles abroad are Marxists, including advocates of violent opposition to the government, or politicians who willfully violated political restrictions.

Human rights advocates respond that many of those abroad are relatives of political activists as well as intellectuals, artists, journalists and professionals who were forced out because of their membership in the once-large leftist political parties.

Of those allowed to return so far, according to studies by human rights groups, as many as a third are citizens living abroad who left routinely without knowing they would not be allowed to return.

These findings and the expulsion of non-Marxist opposition leaders have disturbed even right-wing political sectors that have long supported Pinochet but have recently pressured for an easing of repression, several conservative leaders said.

Following the release of the first list of 125 persons allowed to return, one leading rightist spokesman, Jaime Guzman, summed up much of the sector's reaction when he pointedly observed that the action "must be understood as only the beginning of a process that must continue . . . to confront the problem of the exiles in a more all-encompassing manner."