A Cabinet-level committee yesterday ordered the preparation of retaliatory trade actions directed at Brazil in response to Brazilian refusal to ease restrictions on the import of computers and related technology, a senior White House official said.

In a move designed in part to respond to the drive on Capitol Hill for restrictive trade legislation, the White House Economic Policy Council unanimously ordered a trade working group to come up with retaliatory actions against Brazil by June 25.

Talks with Brazil over the trade barriers were "going nowhere," said the senior official, who asked not to be identified. The policy council, whose chairman is Treasury Secretary James A. Baker III, was told that U.S. computer and information technology manufacturers have lost $1.5 billion in sales between 1980 and 1984 as a result of Brazilian trade restrictions.

The official said such diverse participants in the policy council as the State Department and the Council of Economic Advisers agreed on the need for strong action against Brazil. "That's how mad we are," he said.

The policy council acted as House Democrats were preparing to take to the floor a major trade policy bill that the administration has labeled as protectionist. White House officials said they see little hope of blocking the bill, but wanted to stress, with the Brazilian action, that Reagan also is interested in battling unfair trade practices.

There has been mounting pressure from the Reagan administration against Brazil's 1984 law that bans all imports of small computers for eight years. The Brazilian restrictions were among those targeted by Reagan last September in a series of actions under section 301 of the trade act.

Brazilian President Jose Sarney has maintained the law is sovereign and cannot be altered because of external political pressures. He confirmed this recently by approving long-term policy guidelines for the industry's growth based on the trade barriers.

Brazilian Foreign Minister Roberto de Abreu Sodre said May 8 he was seeking talks on the matter with Secretary of State George P. Shultz. On a visit to Portugal, Sodre said, "We are seeking an immediate dialogue and, if there is a chance for this, there is also the possibility of negotiations within the terms of the law," according to Washington Post special correspondent Richard House in Sao Paulo.

In an earlier exchange of letters with Shultz, Sodre sought to reassure him that the closed-market policy -- in spirit similar to the import restrictions that have been used by Japan -- would not spread from data processing to other industries such as biotechnology, fine chemicals and pharmaceuticals, where U.S. industry has a strong presence in Brazil.

He also said Brazil is willing to accept joint ventures with U.S. corporations in areas where transfer of technology is required. The law says foreign companies can have a 30 percent stake in such ventures if they transfer technology and leave management to Brazilians. But U.S. officials -- in a complaint that was renewed at yesterday's cabinet council meeting -- say Brazil has been blocking foreign investment in industry.

The senior White House official said the Brazilians were not being forthcoming in their approach to the issue. Discussions that began in September were stalled and the Brazilians were using "promises and subterfuge" to avoid progress, he said.

The official also complained about what he described as Brazil's lack of copyright protections.

The White House officials did not specify what retaliatory measures would be drawn up, but they are expected to put limits on imports of Brazilian products such as steel, shoes, aircraft and agricultural items.

Brazil's fledgling data-processing industry is strongly nationalist in tone, in part because it was fostered by former military governments. Brazil currently enjoys a trade surplus of more than $5 billion with the United States, but its refusal to allow outsiders a share of the small-computer market has raised protectionist pressures. There are no restrictions in the market for large computers, which is dominated by International Business Machines Corp.

So far, no major joint venture has been approved by Brazil's Special Secretariat of Informatics (S.E.I.), but IBM already has signed a $20 million agreement with Brazilian steel maker Gerdau Industries to form a new company, Gerdau Informatica, that intends to begin operations in June.

The agreement is seen as a test case of Brazil's willingness to grant concessions, and is feared by nationalist sectors as an opening that would lead to the end of closed markets.

S.E.I. already has expressed reservations about the IBM-Gerdau contract. Gerdau is buying out sectors of IBM's large Brazilian subsidiary that are not restricted under the law. The new company will offer only data-processing services and software and will not be permitted to assemble IBM's personal computers, which have been widely copied in Brazil.

Despite President Sarney's determination to stand behind the 1984 law, there are signs that the nationalist tide could be receding as Brazil seeks to avoid conflict with its main trading partner, correspondent House reported.