Twenty-three Cuban criminals, among thousands who fled their island homeland five years ago by boat, were sent back to Havana yesterday under a U.S.-Cuban agreement that calls for about 2,700 refugees to be deported as undesirable aliens.

According to the Justice Department, four of the deported Cubans were murderers, eight had participated in assaults, eight in robberies and three in drug trafficking.

A civilian chartered Boeing 727 jet carrying the handcuffed refugees left Dobbins Air Force Base in Marietta, Ga., at 3:12 p.m. for Havana.

Agence France-Presse reported tonight from Havana that the Cuban foreign ministry confirmed the plane's arrival and referred to the refugees as "a group of 23 Cuban citizens" returning in keeping with the "normalization of migratory procedures" between the two nations.

Attorney General William French Smith, appearing at a news conference a few minutes after the departure, said the flight opened the way for the resumption of normal emigration from Cuba, which would bring between 20,000 and 30,000 Cuban emigrants to this country legally each year.

The 23 Cubans, who had been held in the Atlanta Federal Penitentiary, were part of an estimated 125,000 refugees who swarmed to U.S. soil in 1980 in a "freedom flotilla" boatlift from the Cuban port of Mariel. He said the "vast majority" of the Marielitos "have incorporated themselves into American life," and are being processed for legal resident status.

But he added that the Reagan administration would "take every step possible" to stop another "immigration emergency" from "ever happening again."

"We say 'yes' to legal immigration and 'no' to illegal entry as a matter of policy of this government," Smith said.

The refugees were herded aboard two buses 90 minutes after the 11th U.S. Circuit Court of Appeals overturned a lower court ruling that had blocked the deportation of 16 of the 23 Cubans.

The 16 had asked that their cases be reviewed "to permit a balancing between the seriousness of the crimes and their claim for asylum."

But the appeals court ruled that federal law "provides that excludable aliens who have committed serious crimes or are threats to the security of the United States may be returned notwithstanding their claims for asylum."

The Reagan administration signed an agreement Dec. 14 with Cuban President Fidel Castro's regime that allows the deportation of 2,746 Cubans at a rate of 100 per month. All are said to have committed crimes in Cuba or the United States, or are mental patients.

In exchange for their deportation, Cuba will allow up to 30,000 people to emigrate to the United States each year.

James H. Michel, a State Department official and author of the Cuban accord, told the appeals court in an affidavit that Cuba could back out of the agreement to accept the refugees if the United States did not keep its bargain to accept the legal emigrants.

Dale Schwartz, an attorney for the Cubans, said he does not plan to appeal the 11th circuit decision. But, he said, "we're going to continue to fight for those fellows that are still left here."

Incarceration of the Mariel Cubans in Atlanta has cost the United States about $41 million annually, according to the Justice Department.