The Federal Trade Commission yesterday voted to expand its internal investigation on a controversial contract granted to a law judge who had handled the FTC's antitrust case involving three leading cereal manufacturers.

The FTC also denied motions by the cereal companies to dismiss the case, start a new trial, remove the judge's successor on the case, set up a special evidentiary hearing or bring in an outside judge to evaluate the case.

In September 1978, the law judge on the cereals case, Harry Hinkes, signed a special contract granting him $72,000 a year to continue hearing the antitrust matter. The previous year, before he retired from the FTC, Hinkes was paid $49,000.

FTC Chairman Michael Pertschuk said later that month that the arrangement by which Hinkes remained on the case was of "questionable validity," said Hinkes was removed from the case.

Two FTC members, Pertshuck and Robert Pitofsky, are not hearing the Hinkes case. Pertschuk removed himself two weeks ago.

"This administrative decision [to approve the contract] was reasonable at the time, but when [the cereal companies] raised serious objections to the plan, the chairman properly decided not the pursue it," the commission said.

"Under these circumstances, we perceive no appearance of impropriety which requires dismissal of the proceeding -- particularly in view of the important public interest in resolving the charges set forth in the complaint," the commission said.