There won't be any fireworks. Any hints of friction will be muted. But, when Secretary of State Cyrus R. Vance meets other NATO foreign ministers in Brussels today, all will be aware that U.S.-West European relations are again beset by strain.

"There's no crisis, but there very definitely is a feeling of malaise and drift," one Carter administration policymaker concedes in talking about the state of the Atlantic alliance.

"What's particularly frustrating about it," he adds, "is that it seems to be rooted less in substantive issues than in nonspecifies - call them vibes or atmospherics that are especially hard to pin down and confront."

His assessment is echoed on both sides of the Atlantic. Diplomatic sources, interviewed here and in major European capitals, agree almost unanimously that an element of armslength wariness has crept into U.S. European relations in the months since President Carter took office.

That, they note, is a cyclical phenomenon that occurs whenever a new administration comes to Washington.

Inevitably, there is a period of adjustment during which the new administration and European leaders must take each other's measure and forge new working relationships.

For many European governments, though, the latest change has been unusually traumatic. For one thing ter replaced an administration with which they felt especially comfortable.

In the post-Vietnam atmosphere of the final Nixon-Ford years, Secretary of State Henry A. Kissinger - a man regarded on the other side as more European than American in his thinking - had put heavy emphasis on relations with Western Europe, gradually restoring them to what the Europeans perceived as a position of primacy in U.S. foreign policy.

Where Kissinger excited almost mythic admiration among the Europeans, their initial perceptions of Carter were quite different. Suddenly, they were dealing with a Washington outsider, surrounded himself with unfamiliar faces and preached exotic new notions like human rights.

For many European leaders, trying to understand Carter hasn't been easy. As the months have gone by, they have remained uncertain about whether there is a grand scheme to his foreign policy or whether it is a collection of romantic ideas unrelated to reality and executed by amateurs.

This sense of unease is by no means uniform throughout Western Europe. In Britain, for example, initial skepticism toward Carter's off-the-wall approach - what one London official cals - foreign policy by press conference - has largely given way to admiration and an increasingly close sense of partnership with Washington.

At the opposite pole is West Germany, where officials of Chancellor Helmut Schmidt's government make little secret of the fact that their feelings about the Carter administration are, at best, mixed.

In France and the other countries, perceptions of the Carter administration seem to follow a more middle course.But, diplomatic sources note, all retain lingering pockets of uncertainty about what Carter is all about.

The strains in the alliance aren't entirely a matter of perceptions and vibrations. There also are a number of very specific unresolved problems haging in the air between the United States and Western Europe.

Chief among them are fears that Washington's move toward a strategic arms limitation talks (Salt) agreement with the Soviet Union could adversely affect the balance of Western Europe's defense and disagreements about whether U.S. economic policy is aggravating European inflation and unemployment.

These have been matters of particular concern to West Germany, whose export-oriented economy is the strongest in Europe and whose geographic location makes it the front line of NATO defenses. In a much-remarked Oct. 28 speech in London, Schmidt zeroed in on both subjects, making bluntly clear his doubts about Washington's direction and its potential effects on Europe.

On the economic side, he warned of "the necessity to safeguard the basis of our prosperity" - free trade, access to energy and raw materials and a sound international monetary system. That was a clear, if implied, reference to Schmidt's belief that the United States is endangering prosperity through its squandering of world energy resources and its failure to resolve balance-of-payment problems that put pressure on European currencies.

Turning to strategic matters, Schmidt said a U.S. Soviet SALT agreement would neutralize the strategic nuclear capabilities of the tow superpowers, thereby putting Western Europe at a disadvantage because NATO is outmanned by the Soviet bloc in conventional forces and outgunned in nonnuclear tactical weapons.

That portion of his speech was an implied rebuke to the Carter administration on several counts: failing to take the interests of its allies into account as it pursues a SALT accord, failing to consult and inform the NATO partners sufficiently about the SALT negotiations and possibly failing in the future to utilize such "gray area" weapons as the neutron bomb as bargaining chips to wring force reductions from the Soviets.

The potential for such substantive differences has always been a built-in part of the alliance. What makes it especially worrisome to many diplomatic observers in the current instance is fear that misunderstandings caused by the Carter administration's style will make them more difficult to resolve than has been the case in the past.

Initially, these misunderstandings centered on some of the newcomers that Carter brought into the U.S. foreign policy machinery.These newcomers, many with ideas forged in the antiwar and civil rights movements of the 1960s, caused fears that Washington would de-emphasize its European ties to concentrate on what the Europeans consider a romantic involvement with the Third World.

In countries like France and Italy, where there is concern about the rise of Eurocommunism some leaders initially feared that the Carter people might start experimenting with closer ties to the West European Communist parties.

The main problem, though, involved human rights and the fear of Europeans that Washington's insistence on pressing that policy would antagonize the Soviet Union and undo the spirit of detente.

As U.S. officials are quik to point out, these fears proved groundless. In regard to Eurocommunism, the Carter administration, while eschewing the shrill hard-line stance of the Kissinger period, has given the Europeans no reason to believe it has any illusions about embracing communism in Western Europe.

Similary, by lowering its voice somewhat on the human rights issue, Washington has shown the Europeans that it can continue the dialogue with Moscow. In the process, many European leaders have been converted to the idea that a concern for human rights is a legitimate, innovative and morally porper subject of foreign policy.

But, while many of the initial doubts have been swept aside, quite a few remain. At present, the most persistent seems to involve the fear - again most prevalent in West Germany - that Washington doesn't place a high enough priority on Europe.

Frequently, this concern is articulated by the complaint of Europeans that there is no "Europeanist" in the top levels of the Carter foreign policy team.

Although Vance has won considerable respect from his alliance counterparts, many remain unsure that he and the President are really interested in Europe. And, they argue, there is no one at the top who seems to be filling the Europeanist role fulfilled in the Democratic administrations of the 1960s by Under Secretary of State George Ball and in the Nixon and Ford years by Kissinger and his key aide, Helmut Sonnenfeldt.

The man the administration considers its top European expert - national security affairs adviser Zbigniew Brezezinski - is a Soviet affairs specialist whom many West European leaders regard as lacking a knowledge of, or feel for, their part of the continent.

Administration officials, while conceding that the proliferation of new faces has made the Europeans nervous about Washington's interest, insist that Carter, Vance and all other ranking administration policymakers are deeply concerned about keeping the Atlantic alliance healthy.

The President's interest, they note, is underscored by the fact that he is about to make his second trip this year to Europe and by still evolving plans to make the spring NATO meeting a heads-of-government summit in Washington.

Even more importantly, the officials add, Vance and others are doggedly pushing ahead to keep up their consultations with European officials and establish the close working relations that will gradually untie the knots of tension in U.S. European relations.

"During the Carter's first six months," one points out, "the main fear in Europe was that we would alienate the Russians and not be able to deal with them. In the second six months, that changed to fears that we were getting too close to the Russians and might make deals with them, over the heads of our European allies."

"That," he asserts, "proves two things' first, that we're never going to have a situation where relations with Europe will be completely harmonious and second, that we've made enough progress to at least show the Europeans that we're not the amateurs and hayseeds they thought we were."