Philip B. Heymann, a Harvard Law School professor who served as a Watergate associate special prosecutor, has been picked to become head of the Justice Department's Criminal Division.

While the appointment has not been officially announced, high-ranking Justice Department officials said yesterday that Heymann had agreed to take the job. He would replace Benjamin R. Civiletti, who has been nominated as deputy attorney general.

The criminal Division slot has been vacant since Civiletti became acting deputy in December. Attorney General Griffin B. Bell has complained publicly that the recent drawn-out hearings on Civiletti's nomination have hurt the department and have delayed filling other vacancies, such as the top criminal post.

Heyman, 45, worked with Watergate Special Prosecutors Archibold Cox and Leon Jaworski, concentrating on the investigation of the White House "plumbers" unit.

He assisted in the trial of White House aide John D. Ehrlichman for his part in the break-in of the office of Daniel Ellsberg's psychiatrist.

Heymann was considered the government's expert in arguing against a "national security" defense in the case.

He defended that argument successfully when the conviction was appealed in 1975.

Before becoming a criminal law professor at Harvard in 1969, Heymann served for a time as a public defender in the District of Columbia. In this capacity he took part in a number of criminal trials, including some involving murder charges.

From 1965 to 1969, Heymann worked for the State Department, and from 1961 to 1965 he was at the Justice Department in the solicitor general's office.

A native of Pittsburgh, Heymann is a graduate of Yale University and Harvard Law School. He is married and has two children.

Contacted at his home yesterday, he declined to comment on the appointment. "I'd rather not say anything for a few days," he said.