Oliver J. Keller, a Democratic member of the U.S. Parole Commission, filed suit yesterday against President Reagan and the Justice Department, charging that the administration is illegally trying to remove him from his job.

Keller, who was confirmed by the Senate for a term that runs until 1984, received a letter last month from Associate Attorney General Rudolph Giuliani saying that Attorney General William French Smith had recommended that he be removed Jan. 15, but that he would be allowed to stay on as a consultant until March 15.

Justice Department officials have confirmed that the administration planned to remove all Democrats from the policy-making commission because the president wanted to work with his "own team."

But at a hearing in U.S. District Court here yesterday, the department backed off somewhat. Paul McGrath, head of the Civil Division, told the court that, in fact, the president has not yet decided to fire Keller and that, if the decision is made, Keller will get 30 days' notice.

The House Judiciary Committee wrote Smith last month, saying his department does not have the authority to remove Keller for political reasons and criticizing the "suggestion that the independence of this important agency should be subject to the partisan whim of this administration or any other."