President Reagan swam and skipped stones in the ocean today while a spokesman made light of the growing budget deficit and Caribbean leaders politely scoffed at Reagan's concern about Marxist expansionism in the region.

For Reagan, after two days of meetings with Caribbean officials, it was the first of three days of fun in the sun.

Accompanied by columnist William F. Buckley and his wife, Patricia, the Reagans splashed the clear, blue waters of the Caribbean near a villa owned by an old Reagan friend from Hollywood, actress Claudette Colbert. The public Cobbler's Cove Beach was blocked off to protect Reagan from being photographed relaxing in a sunny clime while the East suffers through winter-like temperatures.

But the television networks won out this time in the constant long-lens game of hide-and-seek they have been playing with the Reagan presidency. Operating from a cooperative hotel that juts out into the ocean south of the beach, television cameramen filmed Reagan, who in his youth was an accomplished swimmer and lifeguard, as he did the crawl and backstroke, skipped stones and examined a piece of coral.

Offshore, a gunboat patrol and Secret Service men in a dinghy waved away a passing sailboat.

A White House official estimated that the total cost of the five-day "working holiday," which began Wednesday with meetings in Jamaica, would be about $3.5 million. The vast communications and security apparatus that surrounds an American president wherever he goes includes, in this instance, at least three airplanes, several helicopters, a naval communications vessel with hospital facilities and three black presidential limousines, which are conspicuous by their bulk and presence on the island's twisty narrow roads.

White House officials have been especially sensitive about what some of them consider the negative public relations aspects of the president's Caribbean vacation. This concern has affected Reagan, who two nights ago closed a toast to Jamaican Prime Minister Edward Seaga by telling his guest how hard he had worked and saying that "you will find it reported fulsomely that we had a lengthy, leisurely vacation while we were here."

Today, when a reporter managed to get close enough to ask if he were "defensive" about his holiday, the president paused briefly and responded: "No, I'm very pleased to have a day off."

While Reagan was relaxing, deputy press secretary Larry Speakes was fending off questions about revised administration budget estimates showing deeper federal deficits for the next three years. Speakes said the changes were not significant and continued to predict that the economic program ultimately would succeed.

He also said that further discussions between administration and congressional leaders will be necessary to resolve the budget impasse.

Earlier, Barbadan Prime Minister G.M.G. (Tom) Adams gently took issue with Reagan's concern about the threat from neighboring Grenada, whose leftist government has close alliances with Cuba.

Adams said that neither Cuba nor Grenada was a military threat and observed that Barbados has five times the per capita income of Grenada and spends far more on health and education.

At luncheon with East Caribbean leaders Thursday, Reagan had said that Grenada "now bears the Soviet and Cuban trademark, which means that it will attempt to spread the virus among its neighbors."

Adams said he thought that Reagan was speaking for domestic television consumption and said that he instead looked "forward to the spread of the democracy virus from the surrounding islands to Grenada."

Eugenia Charles, prime minister of tiny Dominica, expressed a similar view at a separate press conference. She said the Caribbean nations "shouldn't be confused with Central America because we have different hopes and aspirations."

Leaders of four small Caribbean nations, including Dominica, on Thursday asked for direct aid to help them build roads and develop water and electrical power systems. Both Adams and Charles said they thought the president had been receptive to their views.