In a move symbolic of the new administration's concern about combating communist influence in the Caribbean, Jamaican Prime Minister Edward Seaga yesterday became the first governmental head to visit President Reagan at the White House and urged him to make a "foreign policy thrust" into the Caribbean.

Seaga's 2 1/2 hours of talks with Reagan -- the highlight of a two-day official visit here -- were described by administration officials as underscoring the president's desire to give highly visible backing to those Caribbean leaders identified with policies of support for free enterprise and coolness toward Cuban President Fidel Castro.

Seaga, who won a landslide victory five days before Reagan's own triumph last fall, is regarded as the prototype of the sort of political leader the Reagan administration would like to see setting the pace of political and economic development in the turbulent Caribbean region.

Seaga said the recent shift from leftist to centrist governments in the Caribbean has "presented a timely moment. And one in which a new administration coming into Washington . . . has an opportunity to capture in terms of a universality of thinking and of action."

The Seaga visit illustrates a potentially major shift away from the Caribbean policies pursued by President Carter, whose administration put great emphasis on cultivating the man beaten by Seaga in Jamaica's elections last October, Michael Manley. During his years in office, Manley was a strong exponent of heavy governmental planning and controls over his country's economy, a leader of the nonaligned movement who frequently took positions critical of the United States, and a close friend of Castro.

By contrast, Seaga, while planning to continue Jamaica's nonaligned stance, has moved energetically to dismantle the close ties built up with Cuba by Manley and to attack Jamaica's enormous economic problems by downgrading governmental controls.

Reagan and Seaga conferred privately and attended a state luncheon with American industrialists and bankers, at which Seaga appealed for investment in this country.

"Those here today who believe in free enterprise," Reagan told the lunch guests, "have a great opportunity to put that belief into action."

Alluding to Manley's association with Castro, Reagan and Jamaica's "recent struggles to remain free of foreign interference" have been an inspiration.

Even before Reagan took office, the Carter administration, which had grown increasingly disenchanted with Manley, moved to aid Seaga with a special $40 million aid allotment. U.S. officials said yesterday further increases in U.S. aid are likely under Reagan, although they cautioned that it would be unrealistic to expect the United States to commit itself to a massive Caribbean "Marshall Plan" on the order that Seaga has called for.

Instead, the officials said, U.S. expressions of backing for Seaga are more likely to take the form of encouraging American firms to consider increased investments in Jamaica.

Other topics expected to come up for discussion during Seaga's visit are Jamaica's hopes for expanding its bauxite and textile exports to this country, its concern that the vital Jamaician tourist industry could be hurt by congressional action giving special tax advantages to U.S. organizations holding conventions in Mexico and Canada, and U.S. desire that Seaga take steps to curb his country's billion-dollar illicit marijuana trade with the United States.

Seaga surprised his hosts and guests at the state luncheon by offering the 52 freed American hostages a week's free vacation in Jamaica.