President Reagan's controversial nomination of Ernest W. Lefever as assistant secretary of state for human rights is opposed by Lefever's brother, who said yesterday that he went through weeks of soul-searching about his loyalty to a family member whom he loves but believes to be "unqualified" for the post.

Donald R. Lefever, 57, of Minneapolis, said he finally concluded that his faith imposed a higher duty to "all men and women in the world, of all races," because they are "brothers and sisters" who may suffer from the human-rights policies of his brother, Ernest, and Jeane Kirkpatrick, ambassador to the United Nations.

In an interview, he confirmed that he had made a lonely "low-key decision' about two days ago to phone the office of Sen. Paul E. Tsongas (D-Mass.), a member of the Senate Foreign Relations Committee, to voice his concerns privately. The Washington Post learned of the call from other sources.

He recalled asking Christopher Chamberlain, an aide he did not know, to relay his concerns to Tsongas, but said he didn't urge "outright" that the senator vote against the nominee. Numerous religious and other groups will testify against Ernest Lefever at a confirmation hearing set for May 18 and 19.

Donald Lefever, a manager for Universal Cooperatives Inc., said he had not discussed his decision with Ernest or with his three other brothers, so as not to put them "on a spot."

"It's very difficult to oppose your own brother, to come out against somebody you love and respect" and who could "make a real contribution" elsewhere in government, he said.

At the State Department, an aide to the nominee declined to comment because of a policy not to give interviews until after his confirmation."

Donald said his "basic concern" is that the policies of his brother and Kirkpatrick, who are friends, in "supporting authoritarian regimes in Latin America and elsewhere will tend to repress human freedom for many people." He said he reject Kirkpatrick's argument that authoritarian regimes, unlike totalitarian ones, are only "moderately repressive."

Donald said that in the 1940s, a period when Ernest shared his brothers' continuing pacifist leanings, "Ernest had an enlightened attitude on race and minorities." Asked if that attitude had changed, Donald declined to comment.

"If I were nominated for secretary of defense," he added, "I would expect and hope in all good conscience that Ernest would oppose me."