Five armed federal police agents in Sao Paulo detained Nobel Peace Prize winner Adolfo Perez Esquivel last night and threatened to expel the visiting Argentine human rights activist if he continues to speak out in favor of reparations for Brazilian torture victims.

Perez Esquivel was released after being lectured for two hours, but police drew revolvers and wielded machine guns to control a crowd of prominent Brazilians, including the highly popular archbishop of Sao Paulo, who had gathered to protest outside the police station.

One policeman punched Sen. Jarbas Passarinho, an ex-colonel and a leader of Brazil's government party. Another pulled a pistol, and, standing in front of Cardinal Paulo Evaristo Arns, archbishop of the world's most populous Catholic diocese, threatened: "No one's coming in. If anyone takes a step, I'll shoot."

Today, Perez Esquivel said he intends to continue a week-long speaking tour throughout Brazil, and leave next Thursday as planned. While acclaimed internationally for his years of low-key efforts to protect human rights in South America, the part-time sculptor has been all but ingored by Argentina's military rulers since winning the prize. Several years ago they had jailed him.

Here, the 1980 Nobel laureate ran afoul of a retrictive law passed last year that forbids foreigners from commenting publicly about Brazilian politics. Police say they detained him after a Sao Paulo newspaper published an interview yesterday in which he argued that torture victims who seek reparations for moral and material damages are "not seeking revenge."

In the recently liberalized political climate, President Joao Baptista Figueriredo, a retired general, has ended censorship of the press, and under a general amnesty program has emptied the jails of political prisoners and allowed exiles to return.

But fearful of a purge of the security forces, Figueiredo's more conservative colleagues pledged to support the amnesty only if it would also prevent trial of alleged military and police torturers from the antiguerrilla campaigns largely ended 10 years ago.

Perez Esquivel's remarks crossed a line drawn last week by Brazil's still-dominant military. In three coordinated notes, the ministers of the Army, Navy and Air Force strongly defended those antiguerrilla operations and bitterly accused former political prisoners of "seeking revenge."

Earlier this month, an amnestied political prisoner filed papers suing the owner of a house where she alleged she was tortured for three months during 1971. A former member of the clandestine Revolutionary Popular Vanguard, the militant has acknowledged helping kidnap the Swiss ambassador in 1971.

In another case this month, the head of a Rio group calling for still broader amnesty charged that the current commanding general of the military police here tortured his son to death in 1971.

Last week's military notes were emphatic: "The Army energetically rejects the malevolent insinuations of those who now try to publicly execrate those who fought in true war operations for the preservations of peace, tranquility and the Brazilian family," said one note.

The Air Force minister, considered a relative liberal, admitted: "We were violent. Injustices took place, and we don't deny the errors. But who can be interested in judging an outdated phase?"

The present political "opening," as it is known here, is to lead to virtually unrestricted elections in 1982 for congressmen, governors and mayors. But Perez Esquivel's rough reception comes at a time when many political commentators have said the opening is closing to Brazil's restive labor force.

Last month, federal prosecutors added more charges to a government trial against Luis Ignacio da Silva, president of the workers' party. Known popularly as "Lulu," the barrel-chested labor leader is expected to be convicted next week for leading an illegal 41-day metalworkers' strike in Sao Paulo last March.

Last week, da Silva, returned from a two-week tour to seek support abroad. He met in Rome with Polish labor leader Lech Walesa and Pope John Paul II, with United Auto Workers President Douglas Fraser in Detroit and with AFL-CIO representatives in Washington.

If convicted, the immensely popular "Lulu" will not be able to run for any office in the 1982 elections.