Former secretary of state Cyrus R. Vance took pointed issue here today with the notion that "simply more toughness" toward the Kremlin will compel the Soviet Union to capitulate to U.S. demands.

Shallow debates about U.S.-Soviet relations "based upon slogans and stereotypes" and whipped by "the political winds and passions of the moment" can imperil the nation, Vance warned. In a world of nuclear weapons and great turbulence, he said, "we cannot afford this ignorance."

Vance spoke at the launching of an ambitious commitment to expand American study of the Soviet Union, which has slipped to a level that alarms government and academic specialists.

Columbia University today established the W. Averell Harriman Institute for Advanced Study of the Soviet Union, with a $10 million gift from the former ambassador to the Soviet Union and his wife Pamela, plus $1.5 million from other Harriman family sources.

With this bequest, Harriman, a legendary figure in diplomacy, made a rare concession to mortality. "I'm getting a little older," he said earlier this week at his Washington home in Georgetown. "I'll be 91 next month, you know, and still pretty vigorous, thank God."

Today he added, "I don't see so well but . . . my mind is working," and Harriman said he was anxious to join "in some of the early discussions" of the institute.

Harriman, who first visited the Soviet Union in 1926, said one reason for founding the institute is that there is "so much misinformation" about the Soviets circulating in the United States, "beginning with those in the highest authority of government." He said the greatest "danger in the world comes from the Soviet Union," but that makes it critical to identify clearly where it is possible and impossible to agree.

The new institute will absorb the present Russian Institute at Columbia, oldest and largest center of its kind in the United States, headed by Marshall D. Shulman, adviser on Soviet affairs to Vance in the Carter administration. Shulman becomes director of the new institute.

At a time when Soviet students of U.S. affairs far outpace American students of the Soviet Union, the scholarly impact of the Harriman bequest "is enormous," said Marshall Goldman, associate director of the next-largest Soviet center in the United States, the Russian Research Center at Harvard University.

Some conservatives may construe the Harriman institute as a counterweight in the political-academic spectrum. Since the Reagan administration came to office, conservative "think tanks" and scholarly centers have enjoyed new prestige, influence and access to money. By contrast, Harriman and his wife are both active Democrats, and Shulman and Vance were champions of detente in the Carter administration.

The Columbia faculty, however, also produced Zbigniew Brzezinski as national security adviser to Carter. And Brzezinski is a hawkish counterweight to Shulman, a longtime friend of the Harrimans.

Both Harriman and university officials therefore stressed the total independence and objectivity of the new institute. Columbia University President Michael I. Sovern sought in the dedication ceremonies to forestall any suggestion that the institute reflects any political orientation.

"The institute will not be characterized by hard lines or soft lines or party lines," he said. "Its pledge of fealty is to uncompromising honesty and outstanding quality. It will, in other words, be an institute worthy of the man for whom it is named."

Harriman himself at various times has been criticized as a hawk and as a dove. His exceptional career encompassed service as President Franklin D. Roosevelt's special wartime representative in Britain and ambassador in Moscow; President Truman's ambassador in London and secretary of Commerce; governor of New York; President Kennedy's undersecretary of state; President Johnson's ambassador to the Vietnam peace talks; and a consultant to President Carter for his 1979 summit conference with Soviet leader Leonid Brezhnev.

In his address, Vance said it is vital for American policy to be based upon "realistic and objective knowledge and understanding of the Soviet Union, the Soviet system and Soviet behavior."

Vance said "many people in responsible positions seem to believe that what is called for in our relations with the Soviet Union is simply more toughness." To "embrace a policy of continuing confrontation," he said, "is to confuse firmness with bellicosity," which can only heighten tension, increase the risk of war and "strengthen the repressive tendencies within the Soviet Union" at a time of transition in Soviet leadership.

It is essential, Vance said, "that we leave no room for doubt in the minds of the present and future leaders of the Soviet Union, that the option of moving toward a less dangerous and more constructive future relationship is open."

For its own vital interests, Vance said, the United States must end its "extreme swings in public attitudes about the Soviet Union" and take accurate measure of its "strengths and weaknesses . . . without exaggerating or minimizing either."

Harriman had said earlier that his $10 million to Columbia is his major bequest from the wealth he inherited from his railroad magnate father, Edward Henry Harriman. It is unusual for Harriman to discuss his resources at all.

He said, "I didn't inherit vast sums," in the scale that many assume. His father, he said, left his mother "about $70 million" in 1909 , but "she gave a great deal of money away." Harriman said his mother's estate was divided among her children, three daughters and two sons, with him and his brother, Roland, receiving the larger amounts.

His gift to Columbia, he said, is "a substantial sum" of his resources, although he declined to be more specific. The bequest evolved only in the past year, he said, out of discussion about where to deposit his papers for scholars to use.