A Vietnamese refugee told a House International Relations subcommittee yesterday he saw 46 American prisoners of war in Vietnam as recently as April 1977.

Ngo Phi Hung said he first came upon the prisoners in a building in Hochi Minh City in June 1975, shortly after the city's fall to the Communists. He said there were 49 men, clad only in shorts, lying the floor, and surrounded by guards. Hung was working as a moving man, transporting furniture from the prison - a former U.S. AID office - to the north.

He said the men were moved to prisons in four other cities. He last saw 46 of them - three has died - in a prison in the city of Quangngai, about 400 miles north of Ho Chi Minh City in APril 1977.

The Pentagon has been unable to explain the fate of 697 Americans who have been listed as missing in action in Vietnam. In February 1977, President Carter established the Woodcock Commission to travel to Vietnam and try to locate the men. The commission concluded that it was "probable that no accounting will ever be possible for most of the Americans lost."

The Vietnamese government has maintained that there are no American prisoners in Vietnam.

Members of the subcommittee on Asian and Pacific Affairs expressed skepticism of Hung's testimony. "You say that this prison was right in downtown Saigon," said Rep. G.V Montgomery (D-Miss.). "It is amazing to me that American civilians in the city of Saigon who were free to roam around did not know of this prison."

Through an interpreter, Hung responded that the prison was a "very private" place and that he was the only one who knew the building was being used as a prison.

He said he was able to visit the prisoners after they were moved from Ho Chi Minh City because he had established a "brotherly relationship" with the captain in charge of the prison.