For years, George Costakis' remarkable private collection of the best of early 20th-century Russian art had hung from the walls and rafters of his Moscow apartment. Visitors from all over the world passed through - ambassadors and barons, top Communists and Teddy Kennedy - marveling at works that might have disappeared if not for Costakis, victims of the longstanding Soviet contempt for avant-garde painting.

Costakis knew it could not last forever. He is nearing 65 and scheduled for retirement from his job as a non-diplomatic employee of the Canadian embassy. His family is tired of living in a museum. Although he was born here, Costakis remained a Greek citizen all his life. Now - for a variety of personal reasons - he has decided to leave. But what would happen to his collection?

Represented are the giants of modern Russian painting: Chagall, Kandinsky, Malevich, Tatlin, Klyun, Popova, Ritko - hundreds of works that he had gathered methodically over almost 50 years when hardly anyone else in Russia wanted them.

Taking them with him in any great numbers was out of the question. The odds are that the Soviets would never allow that on principle. Moreover, Costakis says he deeply believes that the collection should stay behind in the homeland of the painters, preserved in some way as the treasure it is.

"All the Rolls Royces, all the servants and fancy houses I could have do not mean more to me than saving my collection," he said yesterday, heaving his thick shoulders.

But where to leave it? How? With whom?

Costakis' dream was that a place might be found for his collection in the new Tretyakov Gallery, under construction for the past 13 years on the banks of the Moscow River as the Soviet Union's largest art museum. The problem was that exhibition of the 20th-century avant-grade was a highly sensitive matter, a step requiring high-level political consideration.

Now at last the decision has been taken - maybe.

Five months ago, Costakis wrote Minister of Culture Pyotr Demichey, proposing once and for all to turn over to the state most of his collection with the implied understanding that the works will be publicly displayed. Costakis also asked permission to take a representative sampling on which he will (no doubt comfortably) support his family - six persons in all - when they go abroad.

After what to Costakis seemed nerve-wracking, long delays, the necessary-approvals were granted last week. Sitting in his apartment yesterday afternoon, the dull grayness of a late winter's day cast aside by the bright and living colors all around, Costakis did his best to be hopeful:

"The ice is breaking," he said. "I believe these paintings will be seen here as they were meant to be."

Indeed, there have been hints that the official attitude is changing toward the 20th-century modernists. Not long ago selfportraits by these masters were included in a show at the old Tretyakov.A Tatlin exhibition was organized at the Moscow Writer's Club. Genius is coming out of the shadows, Costakis said, and he wants his collection to help.

"I consider that in my way I did a great deal for Russia that it did not do for itself by finding these paintings and keeping them," he said softly.

In his letter to Demichev, Costakis sought assurances that, as he puts it, "not another 10, 30 or 50 years would pass before these paintings are put on display." He specifically asked that the works not be sold, as well they might be in the West, should the Soviets ever choose, for millions of dollars.

Costakis was the son of a Greek merchant living in Moscow before the Russian Revolution, and the trading instinct always stayed with him. As a young man he parlayed a gift of rare stamps into a collection of antiques that wealthy families were selling at low prices in the aftermath of the Bolshevik takeover.

After the war he concentrated on paintings, inspired, he said, by their extraordinary quality - and doubtless also by an instinct for what he thought would someday be worth the trouble.

Gradually foreigners began to hear of Costakis' collection, and even some prominent Soviets showed interest. The late culture minister, Yekaterina Furtseva, quietly encouraged him to go on and gave him permission to display small groups of the paintings abroad and even in several Soviet scientific institutes.

The collection made Costakis something of an unofficial celebrity - an invitation of supper and a private tour had panache.

There are - as you might expect - some gray areas. A year ago Costakis disclosed that a large number of watercolors, graphics, drawings and gouaches that he kept in storerooms, including eight by Kandinsky, had disappeared in three separate burglaries and in a fire at his country cottage. He said they had been stolen, and privately he claimed to know by whom.

Some Russians and Westerners smiled knowingly at the report. "Wise old Costakis," they whispered, had sent the works abroad himself as a financial cushion for his future life. The subject brings tears to Costakis' eyes.

"You must believe me," he says, "I am honest to the point of being a fool." If he had shipped them to the West himself, why talk openly about them," the collector sighs. As a man nearing his twilight years who has devoted the great bulk of his energies to wonderful art, Costakis wants - and wants very much - to be remembered as a philanthropist.