Vice President George Bush met with France's new Socialist president, Francois Mitterrand, today and said that the appointment of four Communist ministers to the French Cabinet is "bound to cause concern" in Washington. Mitterrand, in a statement, implied that the issue is one for France to deal with and not the United States.

"Our European allies are soverign nations and the decision on how they are governed rests with their citizens and with their elected representatives," Bush said after 2 1/2 hours of talks with Mitterrand at the Elysee Palace.

"However, the position of the United States on the subject of Communist participation in the governments of our allies is well known. This participation is bound to cause concern, but, having said that, I do want to emphasize that the talks were warm, productive."

Mitterrand had referred to the issue only elliptically, saying, "The policy of France is that of France and will remain that of France."

The unscheduled Mitterrand statement into the microphones on the steps of the French presidential palace as Bush was leaving after lunch was immediately balanced by reassuring presidential words.

"France," Mitterrand said, "is the faithful and loyal ally of the United States and we had many things to say to each other on that point."

Both men appeared to be saying what they considered to be the minimum to meet the expectations of their respective publics. Their own statements and comments by their spokesmen seemed to indicate that far more time had been spent on other matters, especially the effects of high U.S. interest rates.

It was clear from the various statements that both sides had at least tacitly agreed not to let the Communist question get in the way of working together on other matters.

Speaking of the Middle East, for example, Bush said, "It is not an over-statement to say that we found many, many areas of agreement."

Bush spoke this afternoon at the top of the steps leading into the Elysee Palace where the four new Communist ministers had posed this morning for photographs before entering for their first Cabinet meeting.

At the meeting, Mitterrand seated Communist Transportation Minister Charles Fiterman, the senior Communist minister, in a place of honor to the left of the presidential chair at the Cabinet table.

In his remarks after the talks with Mitterrand, Bush broadly hinted that he had extended the invitation Mitterrand was known to have been hoping for to visit the United States this autumn. In addition to meeting Mitterrand at the forthcoming summits in Canada and Mexico, President Reagan hopes there will be "another meeting as well," Bush said.

The Reagan administration had been holding off on issuing the invitation -- most probably for the bicentennial celebrations of the Battle of Yorktown, in which France played a decisive role in the victory of the American Revolution.

Informed sources here had concluded that Washington was waiting for the outcome of the question of Communist Cabinet ministers. Todayhs meetings gave the impression that the Reagan administration has chosen to make the best of the situation by trying to work as closely as possible with the French Socialist government.

Bush nevertheless struck a somewhat more confrontational approach upon arrival at the airport this morning by saying, "I fully expect that both the membership of the Cabinet and the direction of the new French government will be a subject for our discussions today." An Elysee spokesman said that, while he found Bush's frank, open approach to be refreshing, he could not picture a French statesman making such an implied demand for explanations upon arrival in a friendly country.

Asked whether Mitterrand had presented Bush the Socialist justifications for Communist ministers that have become something of a litany here, the spokesman replied that the French president is certainly not required to offer explanations and that he undoubtedly took a far more de Gaulle-like position, an apparent reference to the late president Charles de Gaulle's strong insistence on French independence coupled with faithfulness to France's alliances.

Despite Mitterrand's retience, Bush was treated to the Socialist rationale in talks with other French leaders. In a meeting with Prime Minister Pierre Mauroy after his lunch and post-luncheon meeting with Mitterrand, Bush heard an account of Socialist-Communist relations going all the way back to their split into two separate parties at the Congress of Tours in 1920, according to a Mauroy spokesman.

The reasoning in the Mitterrand entourage is that Socialist strength and Communist weakness is all the more reason to bring the Communists in.

"We have them completely cornered," said a Socialist government appointee. "They are in a nutcracker. What could be better than taking them in when we don't need them?"

The Socialists have also been explaining that there is no risk of the Communists learning French or Atlantic Alliance defense secrets since serious business on such matters is not conducted in the 44-member Cabinet but in small meetings involving the president, prime minister, defense minister and the chiefs of staff. The practice of dealing with sensitive issues in such so-called "restricted councils" was instituted by de Gaulle and followed by his successors.

The Socialists also reject the U.S. argument that the percedent of Communist ministers is dangerous for southern European countries with important Communist parties like Italy, Spain, Portugal and Greece.

One Socialist answer is that the situation Mitterrand has created in France is so unique that there is no reason for it to serve as a precedent. But the same Socialists also argue often in practically the same breath that the Italian and Spanish Communist parties are so far down the road toward becoming normal democratic parties that their evolution should be encourged.

Italian Communist Party leader Enrico Berlinguer, whose party is the most powerful in Western Europe and which has come the closest to sharing power, has often been reported as saying that he, not Italian Socialist leader Bettino Craxi, is "the Italian Mitterrand."