Special counsel Robert B. Fiske Jr. was abruptly ousted yesterday and replaced with former solicitor general Kenneth W. Starr. The move stunned members of Congress just wrapping up Whitewater hearings as well as nearly everyone connected with the investigation.

A three-judge panel of the U.S. Court of Appeals declined to reappoint Fiske under the new independent counsel law, saying he could have a "perceived" conflict because he originally had been appointed to his job by Clinton administration Attorney General Janet Reno.

"It is not our intent to impugn the integrity of the Attorney General's appointee, but rather to reflect the intent of the Act that the actor be protected against perceptions of conflict," the court panel wrote.

Starr's appointment undoubtedly will delay the wide-ranging investigation, already in its 10th month, of President and Hillary Rodham Clintons' Whitewater real estate venture and their ties to Madison Guaranty Savings & Loan. It was uncertain how Starr's appointment would affect the work already done by Fiske, who concluded in June that there was no illegality in a series of White House-Treasury Department contacts on the Whitewater matter. He also concluded that deputy White House counsel Vincent Foster committed suicide in July 1993.

Starr, 48, a conservative, served as U.S. solicitor general in the Bush administration and as a federal appeals court judge in Washington before that. He is highly regarded in legal circles and has been mentioned often as a candidate for the Supreme Court in a Republican administration. He has never been a prosecutor.

Republicans have complained over the past two months that Fiske has exercised virtual veto power in severely limiting the scope of congressional Whitewater hearings.

Fiske is widely respected as a skilled prosecutor and a man of integrity. But members of both parties said yesterday that they welcome his replacement with Starr if it bolsters public confidence in the impartiality of the Whitewater investigation.

Sen. John F. Kerry (D-Mass.) called it "a good move," because it may prevent people from impugning "the integrity of the investigation." But others questioned whether someone with such strong partisan identification could be impartial.

Senate Minority Leader Robert J. Dole (R-Kan.), publicly complained this week that Fiske lacked aggressiveness. Sens. Lauch Faircloth (R-N.C.) and Alfonse M. D'Amato (R-N.Y.) tried to weave criticism of Fiske into the Senate Banking, Housing and Urban Affairs Committee's hearings on how White House and Treasury Department officials responded to a federal investigation that touched the Clintons.

Fiske was traveling and unavailable for comment, but his office issued a statement in which he said he would do everything he could to help "with a speedy and orderly transition."

Bowing to clamor from congressional Republicans, Clinton in January agreed to a special counsel investigation. Because the independent counsel law had expired, the task fell to Reno, who to bipartisan acclaim appointed Fiske. After Congress enacted a new version of the law, Reno asked the court to reappoint Fiske under its provisions.

The news of Fiske's removal shocked the staff of lawyers who signed on to work with him in investigating the Madison thrift and its ties to the Clintons and the Whitewater Development Corp.

"It was like a bolt of lightning hit Little Rock this afternoon," said lawyer Rusty Hardin, one of the senior members of a staff of nearly 40 lawyers and investigators assembled by Fiske. Many joined the probe out of a sense of regard and loyalty to Fiske, and many will leave with him, predicted Hardin.

"There is a big learning curve," said Hardin. "There is no question this will set the pace way back." He described an aggressive, high-powered staff assembled from around the country, many of whom have been working 12-hour days.

"Anybody who thinks Fiske was not a totally independent prosecutor is just flat-out wrong," he said.

Starr's selection is somewhat surprising because he has publicly challenged President Clinton's claim to immunity from lawsuits while he is in office. Starr recently considered filing a friend-of-the-court brief on the side of Paula Corbin Jones in her sexual harassment complaint against Clinton in which she argues against his immunity claim. In fact, Starr recently debated the topic on television with White House counsel Lloyd N. Cutler.

Two of the three judges sitting on the special panel who oversee independent counsels are conservatives. They are presiding Judge David B. Sentelle of the U.S. Court of Appeals for the D.C. Circuit, who was appointed to a district judgeship by President Ronald Reagan, and Joseph T. Sneed of San Francisco, appointed to the 9th U.S. Circuit Court of Appeals by President Richard M. Nixon in 1973. The third judge, John D. Butzner Jr. of Richmond, was named to the court of appeals by President Lyndon B. Johnson.

According to one legal source, Starr was called about two weeks ago by a representative of the three-judge panel to ask if he would consider being on a short list for the position. When Reno originally named Fiske to the job, Starr had been on her short list, along with Fiske and Chicago lawyer Dan K. Webb, a former U.S. attorney in Chicago who is now representing Rep. Dan Rostenkowski (D-Ill.).

In particular, Sentelle, a staunch conservative, had been "boxed into" seeking an alternative to Fiske by several factors, said a conservative Washington lawyer. In passing the new independent counsel statute, Congress had seemed to call on the three-judge panel to rename Fiske, thus challenging the panel's independence, this lawyer said.

"It would look like they were being steam-rolled into {naming} an independent counsel tied to the administration," if they had reappointed Fiske, the lawyer said.

Philip B. Heymann, who as deputy attorney general recommended Fiske to Reno, said earlier this week as rumors circulated about the delay in Fiske's reappointment: "I think it would be mindless folly to appoint anyone other than Fiske. By now, he's undoubtedly deep into the facts of the investigation and has a staff that is also fully informed and working together... . He's a perfect combination of talent, experience and judgment."

Sen. Carl Levin (D-Mich.) added that the new law specifically gave the court the option of either continuing with Fiske or appointing someone new "if the judges thought it would create more confidence" in the appointee. "They exercised the option," he said. "I hope it won't result in extra delay or cost. I'm not critical of the court. I just hope it works out."

Some congressional conservatives have said they are not satisfied with some of Fiske's findings.

D'Amato yesterday called upon Starr to reopen the investigation into White House contacts with Treasury on Whitewater. Based on this week's testimony to the Senate Banking, Housing and Urban Affairs Committee, D'Amato said he is "convinced that certain people who testified before the committee were not truthful."

Administration officials, at every opportunity this week, recited Fiske's conclusions. His staunchest Senate critic has been banking committee member Faircloth, who is particularly doubtful Fiske investigated all the issues related to the Foster suicide.

"My concern is the lack of aggressiveness on the part of Mr. Fiske that I feel in pursuing the Whitewater matter and his many, multiple ties to the various members of the administration. I feel very strongly that he maybe represents more the problem than the solution to clearing up the Whitewater problem."

Faircloth said Wednesday that former White House counsel Bernard Nussbaum recommended Fiske for a job with Iran-contra independent counsel Lawrence E. Walsh; that Nussbaum consulted with Fiske on two high-level Clinton administration appointments and that Fiske worked with Clinton's private attorney, Robert S. Bennett, on a bank fraud case. He also said Fiske's law firm had represented International Paper Co., which sold a parcel of land near Little Rock to the Whitewater partnership.

Staff writers Sharon LaFraniere, Charles R. Babcock, Pierre Thomas, Toni Locy, Kenneth J. Cooper, Sandra Torry and Joan Biskupic and researchers Lucy Shackelford and Margot Williams contributed to this report.