The United States and India appear headed for a messy and damaging nuclear divorce following the failure of negotiations this week aimed at arranging an amicable end to their 18-year-old atomic power relationship.

While the two sides made public only a bland and noncommittal communique yesterday, official sources confirmed that senior U.S. and Indian representatives made no progress in their talks at the State Department Thursday.

The negotiations were the third in a series of fruitless efforts over the past seven months to find a formula for terminating the 1963 pact under which the United States has supplied atomic fuel and know-how for India's nuclear power reactor at Tarapur. India is balking at retaining international safeguards on nuclear fuel the United States has already provided it.

With India telling the United States that time is running out on negotiations, about the only remaining hope hinges on separate visits to New Delhi within the coming month by Assistant Secretary of State James L. Malone, who was the top U.S. negotiator in the nuclear talks, and by Secretary of State Alexander M. Haig Jr.

Malone is to take advantage of a previously scheduled meeting of the U.S.-India Science and Technology Commission Dec. 7-9 to try once again for a break in the deadlock. Haig is scheduled to visit India in mid-December on a trip that will also take him to Belgium, Pakistan, Sri Lanka and Morocco.

Since the Reagan administration's first talks with India on the issue, April 16-17 in Washington, the United States has expressed willingness to cancel the Tarapur pact. The Carter administration, which was divided on the issue, was able to continue selling fuel to India only after presidential intervention with Congress and a narrow victory in the Senate.

At the center of the negotiations are the terms of a cancellation, especially the U.S. insistence that international safeguards should be continued indefinitely on the fuel and equipment already supplied.

India has refused to accept the continuation of international safeguards as proposed by the United States. Citing increased political pressure at home, the Indians reportedly have been hinting at a unilateral declaration renouncing the nuclear agreement on grounds that the United States has failed to keep its part of the bargain.

Such an outcome, in the U.S. view, could be a painful blow to international efforts to police the uses of nuclear fuel and curb the spread of nuclear weapons.

The practical and political consequences in tension-filled South Asia could be serious, adding momentum to a nuclear weapons race between India and Pakistan. The global consequences could be equally dismal because of the precedent it sets: It would be the first case of international safeguards' being dropped after once being in place under a negotiated agreement.

The United States has made it clear, according to reports, that Indian renunciation of the nuclear pact without agreement on continuing safeguards could seriously damage the already troubled relationship between the countries.

Failure in the Indian negotiations could also set back the Reagan administration's policies on atomic weapons proliferation. Though administration officials have argued that they they were "painted into a corner" by the 1978 Nuclear Non-Proliferation Act and the policies of the Carter administration, a bitter nuclear divorce with India is likely to be harmful to any drive in Congress for greater flexibility in the nuclear export field.

Though the main difference has been intractable, progress is reported to have been made in related questions. India is reported to be willing to pledge not to use the U.S.-supplied fuel for a nuclear explosion or to transfer U.S.-supplied fuel or technology to third parties. The United States is reported to have eased its stand on reprocessing of the U.S.-supplied fuel.

Indian officials have expressed confidence they could continue to operate the Tarapur power plant, which provides electricity for a large part of that country, without fuel from this or any other nation that would insist on similar safeguards.