Armed federal agents entered the federal penitentiary in Atlanta at noon yesterday and began taking into custody more than 1,100 Cuban inmates who had controlled the prison for 11 days, loading them onto buses and military planes for transfer to other federal prisons.

After being photographed, fingerprinted and searched for weapons, the first prisoners left on two buses at 1:44 p.m., according to Bureau of Prisons spokeswoman Kathryn Morse.

The Cubans were being processed at the rate of about 60 per hour, and Federal Bureau of Investigation spokeswoman Sue Schnitzer said the operation is expected to be completed early today. When it is, a contingent of more than 400 FBI agents, including the bureau's hostage rescue team and several FBI SWAT teams, are prepared to enter the prison to capture any hold-outs. They will search the burned-out facility, where the prisoners had held as many as 94 hostages before releasing the remaining 89 early yesterday.

Authorities declined to specify where inmates are being sent. But Morse said the Atlanta inmates, who were serving sentences for more serious crimes than the Cubans who had been held at the Oakdale, La., detention center, where rioting first broke out Nov. 21, will be spread among 12 or 13 medium- and maximum-security federal prisons.

The addition of more than 3,000 Cubans from Atlanta and Oakdale will strain the overcrowded federal prison system. "We're already 60 percent above capacity as it is," Morse said. "But we can do it. We have no alternative."

The Atlanta inmates surrendered early yesterday after negotiating an eight-point agreement similar to one reached by the Cubans at Oakdale earlier this week and also endorsed by Auxiliary Bishop Agustin A. Roman of the Roman Catholic Archdiocese of Miami. Justice Department spokesman Terry Eastland said the agreement applies to all Cuban detainees in the United States, not only to those imprisoned in Atlanta.

The agreement between the government and the prisoners provides for "fair review" of whether those who would be deported to Cuba under terms of a U.S.-Cuba immigration pact revived last month should be permitted to remain in the United States. The announcement of the reinstatement of the immigration pact triggered the uprisings in Atlanta and Oakdale.

The agreement with the prisoners also provides for "expeditious review" of whether Cubans who are being detained indefinitely -- most after serving sentences for crimes committed in the U.S. -- are eligible for release.

Cubans who participated in the riot will not be prosecuted, "except for specific acts of actual, assaultive violence against persons or major misconduct." The agreement specifies that "this does not include mere active participation in the disturbance, failing to depart Atlanta penitentiary during the disturbance, or acts causing property damage."

Attorney General Edwin Meese III defended the agreement from the steps of the penitentiary last night, saying that it did not represent capitulation to the hostage-takers. "You will learn as we proceed in this matter . . . in no institution will people be able to gain by an uprising," he said.

Meese, Bureau of Prisons Director J. Michael Quinlan and FBI Director William S. Sessions toured an area of the facility that is now controlled by the government, and Quinlan said he thinks that the prison will be back in operation soon.

"This prison will be here many, many years," Quinlan said.

Meese said he was proud of how officials dealt with the uprising and added that its cost had not been determined. "Cost was not a factor whatsoever. What was needed was done. The first concern throughout this matter has been the safety of those held hostage," Meese said. An estimated 1,500 officers worked during the siege.

President Reagan commended Meese for his handling of the prison disturbances after Sen. Richard C. Shelby (D-Ala.) called yesterday for Meese to resign. Shelby said Meese "botched" the prison crisis by failing to take precautions when the reinstatement of the agreement to deport the Cubans in the prison was announced and by negotiating with the inmates once they took over the prison.

The Atlanta agreement specifies that Cubans have the right to seek entry to a third country and promises quick review of those who gain admittance. But several federal officials said it is unlikely that another country would accept a convicted Cuban.

Cuba and the United States agreed in 1984 to the immigration pact, which permits 20,000 Cubans to enter the United States annually on the condition that Cuba accept the return of about 2,500 Cubans who came here in the 1980 boatlifts from the port of Mariel, Cuba, but the pact was canceled by Cuban leader Fidel Castro several months later after the United States started directing broadcasts toward Cuba over Radio Marti.

INS Commissioner Alan C. Nelson praised the original immigration pact with Cuba yesterday.

"Many thousands of Cubans can now be reunited with their families in this country through normal immigration," Nelson said. "Additional political prisoners can come out of Cuba; these are people who have been in jail there for 20, 25 years. . . . More will come out as Cuba accepts its obligation as all countries in the world to take people back who are not properly here."