Though House Speaker Thomas P. (Tip) O'Neill Jr. (D-Mass.) and his aides are working hard to smooth over the recent rift with the White House, congressional sources say the relationship between the Carter administration and key Hill leaders was disintegrating anyway, and will not be easily repaired.

The overt cause of the rift between Carter and O'Neill was the ouster of O'Neill's friend, Robert T. Griffin, as the No. 2 man in the General Services Administration. That led O'Neill to says Griffin was "shabbily" treated by the White House and to bar Cartrer chief congressional liaison aide Frank Moore from his office.

Though O'Neill now says it's over and promises that the incident will not affect his handling of Carter's legislative programs, aides still say "no comment" on whether O'Neill will accept Moore's request to meet with him.

But Hill sources also say that was, no pun intended, just the tip of the iceberg in recent relations between the Hill and the White House.

"Something snapped about two months ago," a source close to the leadership said.

"We used to stay, he'll learn, he'll learn, now we just say what next," another source said. "I think we've come to accept that things are never going to normalize politically with this president and his staff."

They use the O'Neill-Griffin-Carter affair as a case in point. Griffin was given another job as an assistant to Carter's trade specialist, Robert S. Strauss. If Griffin were going to be given job, why wasn't that done first? The rift could have been avoided. O'Neill could have been pleased with the "promotion," and the result - getting Griffin out of GSA administor Jay Solomon's hair - would have been the same.

Hill sources say they believe it was part of Carter image-polisher Gerald Rafshoon's campaign that led him to open confrontations with O'Neill and with Sen. Edward M. Kennedy (D-Mass.) over health insurance, and to leak stories of chewing-out that Senate Majority Whip Alan Cranston (D-Calif.) received at the White House.

But they see it as a dangerous game, particularly at a time when Carter's major legislation is heading into the homestretch on the Hill.

"If you're going to flex your muscles, you better be strong," one source said. "Otherwise, you're not fooling anybody, and you're just going to get a punch in the nose in return.

A leadership source said many members of Congress are out polling in their own districts, finding out not only where they stand with their constituents, but where Carter stands, too. "The result are horrendous. Polls are showing him with as little as a 25 percent approval rating." If Carter is trying to appear strong by confronting Hill leaders and taking his case to the media, "nobody cares, because Carter isn't popular in the country. They're not taking him seriously," the source added.

Members of Congress were telling O'Neill on the floor that he should be relieved to be shed of the burden of being "Carter's man."

Relations between the White House and Majority Whip John Brademas (D-Ind.) were strained when Carter came out strongly for lifting the arms embargo on Turkey, a move Brademas, of Greek ancestry, deeply and emotionally opposed.

That leaves Majority Leader Jim Wright (D-Tex.), who led the White House fight for the lifting of the embargo, as the only member of the top three leaders in the House on good terms with Carter.

Hill sources could not put their finger on when relations snapped. "It was just a cumulative thing. The administration's handling of the tax bill, the handling of the Soviet dissident situation. Andy Yound, Peter Bourne, Midge Costanza, a lot of screw-ups large and small around here," a source said. He said staff and members seemed to just reach a point where they "ran out of hope."

Asked where that left Carter's legislative programs, one aide said, "We'll do what we have to do - energy, taxes, appropriations, CETA [jobs bill], maybe civil service reform and airline deregulation, and then we'll go home - and weep."