Gov. James B. Longley yesterday agreed to the proposed settlement of the Maine Indian land claims - which originally had sought three-fourths of the state - thereby paving the way for congressional action.

The move came after a breakfast in the governor's mansion attended by Sen. William D. Hathaway (D), who engineered the settlement, and Republican Reps. William S. Cohen and David F. Emergy. The state's senior senator. Edmund S. Muskie (D), was in Rome for the installation of Pope John Paul II.

Everyone has agreed to the proposal with some details to be worked out," said Attorney General Joseph E. Brennan.

Under the terms of the compromise worked out by Hathaway, who had obtained approval for it from President Carter, the Passamaquoddy and Penobscot tribes would receive $27 million in cash. In addition they would get $10 million that would be used to purchase 100,000 acres of timberland from the state's large paper companies.

Hathaway said Congress could begin hearings on the $37 million settlement proposal in February or March.

Cohen, who originally had opposed any settlement with the Indians, and argued for a test in the U.S. Court of Claims, said he grudgingly endorsed the plan but would rather have seen it settled in court because of the precedent it would set for several states facing similar claims.

"Each of these other cases will look to the Maine case," Cohen said. "It's a question of the price tag and the federal treasury."

Tribal leaders have not formally agreed to the proposal, but the governors of the two tribes said there is an excellent chance their people will accept it.

Approval would end the 10-year squabble between the state and the Indians, who claimed 12.5 million acres of their land was taken from them by Maine - and Massachusetts before Maine became a state - in violation of the Nonintercourse Act of 1790. This law provides that all land deals with Indians must be ratified by Congress.

The federal government had filed suit against the state on behalf of the Indians and trial would have begun in federal court in Portland in March.

Most legal and financial experts had predicted economic chaos in the state if the case went to trial since land titles involving about 330,000 small and large landowners in the disputed area would have come under a cloud.