Soviet military activity around Poland has sharply decreased in recent days as Warsaw Pact troops have returned to their garrisons, Reagan administration officials said yesterday.

A new downhill glide in the Washington roller coaster of concern about Poland was acknowledged in statements from the White House and State Department. Official pronouncements in both places added, however, that Soviet forces are still in position to intervene with little warning, and thus a degree of U.S. concern remains.

At the same time, the State Department expressed concern about Soviet activities in another sector: the planned visit of three Soviet warships and a tanker to Cuba, the Caribbean and Central America. Spokesman Dean Fischer said any Soviet effort to expand its influence in the Western Hemisphere, especially the Caribbean, "only serves to heighten tensions."

The last reported visit of Soviet warships to the area was in August 1979. After public and private U.S. expressions of concern, those ships left the Caribbean without visiting Cuba or approaching Central America.

Such a turnaround is considered unlikely in the present state of Soviet-American relations. State Department sources said the Soviet ships are expected to dock today at Havana and stay at least 10 days before moving on to ports in such places as Nicaragua and Grenada.

Regarding Poland, U.S. officials now believe the latest high point of danger of Soviet intervention actually was reached about 10 days ago. An airlift of Soviet heavy equipment, which is believed to have carried a fresh Russian helicopter regiment into Poland, was among the indicators that spurred U.S. concern to a very high level on Friday, April 3.

On that day Secretary of Defense Caspar W. Weinberger sounded a public alarm by telling a congressional committee that the situation around Poland had been worsening hour by hour, and Secretary of State Alexander M. Haig Jr. seriously considered cancelling his planned trip, starting late that day, to the Middle East and Western Europe.

Beginning sometime on the following day, according to sources here, the tension level began to ease. The first sign was the sudden postponement of an important session of the Polish parliament, followed by the surprise trip to Prague of Soviet President Leonid Brezhnev, who carried a relatively moderate statement on Poland.

Shortly thereafter, the Warsaw Pact announced the end of the Soyuz-81 joint military exercise which had generated Western concern. And now, according to reports, the units engaged in the military maneuver have abandoned their field positions to return to barracks.

Despite all this, the consensus among American officials is that the "Polish crisis" is far from over, because the underlying causes of tension and friction within Poland and with the U.S.S.R. remain unresolved.

Polish Premier Wojciech Jaruzelski, in this view, has once again been granted additional time by Moscow to bring Poland's policies into line with those elsewhere in the Soviet Bloc. But this is seen as only a temporary respite, sources said.