Fidel Castro said today that he will let the Cuban families of more than 80 U.S. citizens living here leave the country.

The announcement on the Cubans - spouses and children of Americans who want to leave Cuba but have stayed for more than 16 years because their families have been barred from going - was made at the conclusion of a four-day visit by Sen. Frank Church (D-Idaho).

Church described the Cuban action as a "very important humane gesture of goodwill on the part of President Castro."

Church said Castro was so accommodating on the question that he offered to send the families home aboard Church's Air Force jet if the Americans could provide a list of who they were.

Since that was not possible, Church said, the arrangement for their departure from Cuba with their families should be made within a few weeks.

Standing beside Castro at an airport press conference before his departure, Church said he and the Cuban leader had also discussed the seven American political prisoners hled in Cuban jails, as well as the Cuban military presence in Africa. No announcement was made on those subjects, and both men refused to comment on them.

Castro "has given me his views" on the matters, Church said, "but these are not kinds of problems that can be solved at this time. I intend to report President Castro's views" about the prisoners and Cuba's role in Africa "to President Carter, but we are not prepared to discuss them at this time."

The action on the Cuban families, however, is likely to be viewed as an important step in the continuing exchange of concessions between Cuba and the United States.

While Carter has particularly emphasized the political prisoners, he has said that the general problem of American citizens detained in Cuba for one reason or another has been a major stumbling block to normalization of relations.

Asked what he thought is the principal remaining issue between his government and Washington, Castro said "time."

Castro, said he thought Church's visit had been "most important" and had contributed to the creation of an atmosphere for the effective development of the process improved relations."

Asked about the prisoner and troop issues, Castro said: "We spoke about different topics. These are complex issues and it is not the most adequate thing to do approach them in a press conference." But Castro said, there is "no topic of conversation banned" with the United States.

The families Church said, had been a "matter of much concern to the U.S. government and the American people. President Castro indicated to me that he thought this was a humanitarian question to which he was very sensitive."

Church said that the families are "all free to go" immediately and that Castro had agreed that an American plane could be sent for them at any time.

Between 80 and 100 families here are affected by the agreement, most American men with Cuban wives. The Americans have always been free to leave Cuba, Church said, but refused to go without their wives and Cuban-born children.

Observers here described the Cuban concession as a major good-will offering on Castro's part, particularly because it comes at what is usually the worst time to expect concessions - immediately following renewed criticism of what the United Stated feels are human rights violations in Cuba.

Last month Carter said he did not see any possibility of normal relations with Cuba in the foreseeable future because of the unresolved issue of U.S. citizens here, both in and out of jail.

With a few exceptions, Castro has refused in recent years to allow Cubans to emigrate to the United States.

Leaving reporters and some staff members behind at the end of a long driving tour in Havana yesterday. Castro and Church spent the evening alone in discussions at a secluded government house on the Cuban coast. This morning the two men went spear fishing in the Caribbean.

"I hoped to make President Castro's acquaintance in Cuba," Church said. "He was given me three days of his time, and I leave Cuba to the behalf that I have found a friend."

Although Castro begged off an earlier promise to conduct a press conference aboard a U.S. Air Force jet that came from Andrews Air Force Base to pick Church up, he did board the plane for a tour without reporters, sat down for a few minutes with Church, and distributed cigars to the crew.