The huge debt owed by Third World countries "will never be paid back," and even the interest on it now being transferred to the richer industrial nations must be sharply reduced, former West German chancellor Helmut Schmidt yesterday told the opening session of a two-day conference here on the future of the World Bank.

He called for the cancellation of the debt of the least-developed countries, but not in the others, fearing that they would abandon necessary efforts to make economic adjustments.

Both Schmidt and New York banker Felix Rohatyn also called on Japan to take a new and more important role in providing funds for financing development needs in the Third World. Rohatyn said that Japan could "easily commit $50 billion over the next five years" to the capital of the World Bank and other multilateral development banks.

Schmidt said that Japan, which has gained "an enormous edge" over its trading competitors through a high savings rate and low military-preparedness expenditures, should boost its official development aid 400 percent "in the next couple of years, and even more later on."

He estimated that Japan now contributes only 0.4 percent of its gross national product in official development assistance, and said that number should be almost quadrupled to 1.5 percent of GNP.

The conference -- attended by, among others, World Bank President A. W. (Tom) Clausen, and by former congressman Barber Conable, who will succeed Clausen next Tuesday -- was organized by the Overseas Development Council, a Washington research group, to help define a new role for the bank in the next decade.

Schmidt said the leadership for a move to reconsider the whole Third World debt structure could not come from Europe, which is divided by too many currencies, tax and regulatory systems. Instead, he nominated U.S. Treasury Secretary James A. Baker III as the logical person to frame a proposal for the 1987 Economic Summit involving new loans from the major countries, as well as outright cancellation of the debt of the poorest debtor countries.

He did not define which countries should receive total relief of their debt burden, but said the new initiative would also have to contain an outline of legislation to alleviate the problems of commercial banks whose balance sheets would be hit by debt cancellations or postponement of interest payments.

"I think there is no doubt at all that the credits advanced to the Third World will never be paid back, and can never be paid back. And there ought to be no doubt that even the interest cannot be paid to the degree which most of our banks are expecting now," Schmidt said.

In a related proposal, Rohatyn -- who addressed the conference dinner last night -- eschewed outright debt cancellation, but urged substantial interest rate relief as well as long stretch-outs of debt, along with government guarantees.

"In order to avoid a massive write-down and impairment of their capital," Rohatyn said, "the banks will require some form of government guarantee, or an exchange for new types of securities if interest rates are reduced and maturities are stretched out significantly." The guarantee could be by the United States or by the World Bank, Rohatyn said.

The sharp reduction in the annual cost of servicing the debt, coupled with the fresh capital provided by Japan, would lower the internal political pressures in major Third World countries such as Mexico, and provide the basis for renewed trade with the West, Rohatyn said.

But Baker and other U.S. officials have reiterated their opposition in the past to write-down of loans, or involving the government in such guarantee programs.

Schmidt acknowledged that the concept that the entire Third World debt cannot and should not be repaid is a difficult one for bankers, the public, and political leaders to accept. But he argued that it is essential that the current transfer of funds from the developing nations to the rich nations -- estimated by the World Bank to have been $22 billion last year -- be stopped