The Justice Department has backed off an unpopular proposal to change the way refugees are granted political asylum following claims by more than a score of immigrant advocacy groups and lawyers that the changes would deny due process to refugees, officials said this week.

The controversial proposal, as initially presented two months ago, called for eliminating immigration judges from the asylum process and instead leaving asylum decisions to a special corps of officers within the Immigration and Naturalization Service.

But following several hundred responses critical of the proposed changes, Justice officials said they will continue to let immigration judges hear the asylum cases of refugees who are in deportation proceedings.

Under an interim procedure in place for the last seven years, refugees get two chances to ask for political asylum: once before the INS and, if they are denied at that level, before an independent immigration judge.

"The more we thought about it, the more we realized that the comments were not without merit," said Roger Pilon, director of the Asylum Policy and Review Unit in the Justice Department that drafted the changes. "We have taken very seriously their points about the elimination of immigration judges from the asylum process."

Pilon said the Justice Department would continue with plans to create the specially trained corps of asylum officers in the INS to conduct "nonadversarial" hearings in an effort to streamline the asylum process.

He said the Justice Department will publish the revised changes in the next two weeks and sometime later would come out with a final plan.

About 30,000 refugees who apply for political asylum every year would be affected by the proposed changes.

"This is something we were very determined about and we thought we were in the right and they have acknowledged that," said Warren Leiden, executive director of the American Immigration Lawyers Association (AILA), which two months ago sent letters to 2,500 members opposing the asylum changes.

AILA and Rep. Joe Moakley (D-Mass.) had criticized the proposal on the grounds that it would deny due process to asylum applicants by leaving the decision to an asylum officer who works for the agency that deports illegal immigrants.

The lawyers and Moakley also had feared the proposal was an attempt by the government to selectively grant asylum to persons fleeing communist countries while denying asylum to those fleeing countries friendly with the United States.

Before Congress enacted the Refugee Act of 1980, only refugees fleeing from communist countries or from religious persecution in the Middle East were eligible for asylum. The act was Congress' attempt to neutralize the political bias by making anyone with a well-founded fear of political or religious persecution eligible for asylum, INS officials said.

But the lawyers argue that even with the current system of granting political asylum, refugees from communist countries still are more likely to receive political asylum than are refugees from noncommunist countries.

Between October 1986 and June 1987, 85 percent of asylum claims filed by Nicaraguans were granted, but only 5 percent were granted for Salvadorans, according to preliminary INS statistics that do not include denials from the Miami INS office. Overall, the INS grants between 25 and 30 percent of asylum requests.

The leaders of Nicaragua's Sandinista government, which won power in a 1979 revolution, are acknowledged Marxists with ties to Cuba and the Soviet bloc. El Salvador's president, Jose Napolean Duarte, is leader of that country's moderately left-of-center Christian Democrats.

While the lawyers said they are relieved that the immigration judges will continue to have a role in the asylum process, they are still concerned about the role and purpose of Pilon's newly created office to review asylum decisions reached by INS officers.

The new office, called the Asylum Policy and Review Unit and created in March by Attorney General Edwin Meese III, already has begun reviewing all denials of asylum claims, Pilon said.

Some AILA members claim the new office is the Reagan administration's attempt to return to the old system of granting asylum only to persons from countries unfriendly with the United States.

But Pilon, formerly policy director for the State Department's Bureau of Human Rights and Humanitarian Affairs, denies his office is politically motivated. He said the proposed system would be more streamlined and efficient because specially trained asylum officers will have more knowledge of conditions in the refugee's country than an immigration judge who now is burdened with many types of immmigration cases.