The U.S. Court of Appeals here yesterday opened the way for federal construction of a $500 million irrigation project in North Dakota, turning back environmentalists and striking down a lower court ruling that had held the project in check.

The three-judge panel ruled that Congress needn't reauthorize the Garrison Diversion Unit, a massive irrigation project that Congress first approved in 1965.

The judges ruled that Interior Department officials are no longer bound by an agreement reached during the Carter administration with the Audubon Society that had kept the project in abeyance for five years.

Under the agreement the department was forbidden from going ahead with the project until certain environmental studies were completed and Congress reauthorized it.

The court declined to rule on the broader constitutional question of whether Reagan administration officials could be held accountable under an agreement reached during the Carter administration.

It sent the case back to U.S. District Court for a decision on the merits of completing the Garrison water system, which Audubon officials argue would pollute fisheries and destroy vast areas of natural wetlands.

U.S. officials for several years have been negotiating with Canadian officials over possible effects on rivers that run into Canadian territory. As a result, Interior officials say, the ultimate size of the project will probably not exceed 100,000 acres, of which only a portion has been completed. The major purpose of the project is to irrigate 250,000 acres of North Dakota.