Attorney General Griffin B. Bell, apparently stymied in attempts to fill the Justice Department's second-ranking position, named Richard L. Thornburgh yesterday as acting deputy attorney general.

The appointment was somewhat unusual because Thornburgh, who has been assistant attorney general in charge of the Criminal Division, has been slated for replacement by the Carter administration.

Justice Department sources said Thornburgh's new assignment was dictated by the need to prevent work from piling up in the deputy attorney generals office while the hunt for a permanent occupant goes on.

Reliable source have reported that the Carter administration's first choice for the job. U.S. District Judge Frank M. Johnson Jr. of Montgomery, Ala., has refused unequivocally. President Carter and Bell reportedly are now seeking someone with strong law-enforcement credentials for the deputy post.

Although a public announcement has not been made, it is known that Bell plans to replace Thornburgh in the Crimincal Divison with Benjamin R. Civiletti, a Baltimore attorney.

Thornburgh is understood to have agreed to remain at Justice in a special capacity for a few months to complete certain investigations under his control. Last month, he served five days as acting Attorney General in the period between Edward H. Levi's resignation and Bell's confirmation by the Senate.

Justice Department sources also said yesterday that Carter and Bell have decided to retain Norman. A Carlson as director of the Bureau of Prisons adn Gen. Leonard F. Chapman Jr. as commissioner of the Immigration and Naturalization Service.

Other sources said, however, that Chapman may be succeeded by Mario Noto, a Washington immigration law-year who was once an INS official.

The sources added that Richard W. Velde, who heads the Law Enforcement Assistance Administration, is slated for replacement but has been asked to stay for 60 days.