President Reagan yesterday announced he will remove Arthur Flemming as chairman of the U.S. Civil Rights Commission by nominating Clarence Pendleton, a conservative black Republican and friend of White House counselor Edwin Meese III, to take his place.

The president also nominated Mary Louise Smith, a former chairman of the Republican National Committee, as vice chairman of the commission to replace Stephen Horn.

Civil rights advocates and commission staff members charged that the action threatens the independence of the bipartisan unit. The only other time a president has removed a member of the Civil Rights Commission was when Richard Nixon dismissed the Rev. Theodore M. Hesburgh in 1972. Hesburgh was critical of President Nixon's civil rights policies.

Flemming, a prominent Republican, a former secretary of health, education and welfare, was appointed to the chairmanship in 1974 by Nixon. Flemming said yesterday that he had no desire to leave and, in effect, was fired. He was notified yesterday morning by Mike Farrell, an aide to E. Pendleton James, the White House director of personnel.

Deputy White House press secretary Larry Speakes gave as the reason for Flemming's dismissal the president's desire "to have his own appointee in the post.

"Mr. Flemming has served for a number of years with honor," Speakes said. "He is a distinguished public servant, but the president felt that he would like to appoint Mr. Pendleton . . . who has an outstanding record in the field of civil rights."

But some civil rights advocates and some members of the commission's staff suggested the White House replaced Flemming mainly because it has been irritated by the commission's positions in favor of voting rights, affirmative action, busing to achieve racial integration and recent reports critical of administration budget cuts and of police brutality.

But some civil rights supporters have also lamented the commission's lack of power and resources and its constant need to walk a in political minefield in order to stay alive. It is mainly a civil rights monitoring agency.

The commission's charter will expire in fiscal 1983, and Sen. Orrin G. Hatch (R-Utah) indicated in earlier appropriations hearings that he might seek to abolish it then. Hatch could not be reached for comment.

The White House choice for the key post of staff director under Pendleton is said to be Alfred Balitzer, a political science professor at Claremont Men's College in California who is working for the Republican National Committee as a liaison officer to minority and other interest groups.

"The fear is that we'll now have a distant and inexperienced chairman, along with a strong and present staff director who will either work to turn the commission around to please Hatch and company in its approach to issues or to prepare the commission for going out of business in 1983," said one staff member.

While Flemming has maintained an office at the commission, Pendleton said yesterday he will continue to work and live in California, commuting to Washington as necessary.

The normally low-profile Flemming yesterday responded to the news by charging that the administration's actions "endanger the independence" that has been the commission's hallmark for 24 years.

He said there is a pattern taking shape in the nation that is "harmful" to the cause of civil rights, one to which Congress and the executive branch have contributed. They are both, he said, "headed in the wrong direction in that they are trying to eliminate or weaken" the measures needed to "take the rhetoric of the Constitution and translate it into reality as far as the lives of people are concerned."

Flemming said he felt he had been present at the birth of the commission, which took shape while he was a member of President Eisenhower's Cabinet.

The Leadership Conference on Civil Rights praised the commission's record of integrity in "pointing out the shortcomings of federal civil rights programs and policies," and called the removal of Flemming and Horn "a clear indication that this administration is unwilling to tolerate independence or dissent."

The NAACP said through a spokesman that it views the dismissal of Flemming with "dismay," because he has served with distinction and is a "good Republican."

Pendleton, president of the Urban League of San Diego, Calif., said yesterday he would stand on his civil rights record. "I don't equate conservatism with racism. I think a lot of people are using that as a straw man . . . .