President Carter gave up his effort to get "instant termination" of West Germany's $5 billion nuclear power deal with Brazil early in his administration, a high U.S. official said today.

The decision was made after the first two weeks, the official said. "No one now expects the Germans to back down," he added.

Another official said that the issue concerning the West German-Brazilian deal had "pretty well played itself out" before the economic summit meeting that ended here Sunday night. At the conference, both officials said, Carter had merely sought, successfully, to start a study that might lead to international control of such facilities.

The administration now hopes that the West German-Brazilian facility "can be diverted to an international framework" in accordance with some future plan that might come out of the study.

The West German-Brazilian deal appears to be recognized as a fait accompli. One official pointed out, however, that it "is a protracted long-term plan over a number of years," and that "it could be subject" to whatever international controls might arise out of the summit agreement.

The American opposition to the deal had raised difficult problems with Brazil and added to a tension with West Germany that also spread to economic issues.

Meanwhile, it was clear from interviews today with a number of Americans and Europeans not only that Carter had gotten along well with West German Chancellor Helmut Schmidt at their first meetings, but that the two-day summit that concluded yesterday had been something of a personal triumph for Carter.

The other leaders took his measure, and they were impressed," said one official. "It was very visible." European doubts about Jimmy Carter, earlier considered a question mark against a more predictable Gerald Ford, seem to have been resolved.

U.S. West German tensionns deepened early this year when Bonn was pressed during Vice President Walter B. Mondale's post-inaugural trip to boost its economic growth rate. A high American official said today that this effort by some sub-Cabinet officers might be excused as an effort to "dramatize" the need for growth.

"But if it was expected that governments would change their goals as a result of the pressure, it was a mistake. It should have been ended a long time ago," the official said.

The flap over nuclear energy arose when Carter took issue with Bonn's 1975 contract with Brazil for eight nuclear power stations, including technology for fuel-reprocessing and uranium enrichment.

These processes could yield material for atomic bombs, unless tightly controlled.

In Europe, which has to depend more and more on subsitutes for fossil fuels, the American position has been viewed as self-serving, because the United States is one of the major sources for highly enriched uranium.

The even-nation summit, after a somewhat tense three-hour session, came to an amicable if still vague agreement on nuclear energy under which the problem will be the subject of a two-month study to establish guidelines.

There was no mention in the final "declaration" of the U.S.-West German flap over the Brazilian deal, but one American official said today that Carter had never intended to force the West Germans to back off.

"He wanted the other countries to understand why there had been such a fuss about the West German nuclear deal and why he was taking certain positions on reprocessing, and on enriched uranium and all of those things."

Carter convinced the other leaders that his effort was not designed to keep a technological advantage for the United States," and that he wanted to work with them in solving the problem.

The Europeans responded that without adequate coal and other energy sources to substitute for oil, they would have to rely more on nuclear power.

Carter "learned something," one source said. The President acknowledged that it is a tough political problem that would require a mutually acceptable solution.

European sources agreed that a way seems to have been found out of what had seemed a tense U.S.-West German situation. The Brazilian deal will go forward. The West Germans, meanwhile, had indicated earlier that they would not make similar deals if other nations also refrain.

The summit-chartered study may provide the framework for a common stand. Meanwhile, Carter's approval last week of renewed shipments of highly enriched uranium to West Germany and other European countries eased another element of the tense relationship.

Carter appeared to have scored as well with the other politicans at the summit as he did on his whirlwind tour into Newcastle and environs last Friday. A West German minister turned to his American counterpart after Carter's brisk and newsy summary of the summit agreement Saturday night, and said:

"Now I know why Carter won the election."

Europeans are looking forward with less apprehension now to the next summit meeting, which is to take place in West Germany early in 1978.

Carter had been an enigma to most Europeans, many of whom frowned on his election-campaign television image as a smiling, toothy Southerner with a less-than-sparkling personality.

When they finally met Carter, they found him not only a charming and successful politician - which he proved on his whirlwind tour into Newcastle and environs - but a sharp, well-briefed, polite but no-nonsense head of state.