Federal Trade Commission Chairman James C. Miller III yesterday called for an easing of restraints on hostile corporate mergers, blaming the existing regulatory system for some management defense tactics.

Miller said the problem was not the number of takeovers, but certain forms of management resistance. Tactics such as paying "greenmail," or huge ransoms to speculators, and using "shark repellants" -- actions to discourage raiders by making takeovers more difficult -- are "manifestations" of the regulatory system, he said.

"We, as a society" should consider either lessening the restraints on takeovers that enable such tactics, or regulating the resistance to takeovers, Miller said in a meeting with reporters, adding that he would prefer the first route.

We "need a competitive market for assets and capital" to "keep management efficient, to keep managers on their toes," said Miller, echoing arguments expressed by the Reagan administration in recent congressional testimony.

Some critics of management defense tactics argue that they enrich the wealth of individual raiders at the expense of other stockholders, and may prevent mergers that would benefit consumers. Critics of the takeover artists argue that they divert capital from more productive uses, and divert management's attention from long-term goals to short-term threats.

A House subcommittee held hearings recently to examine the arguments and to consider whether a legislative response is needed.

Miller cautioned that regulating takeovers lies beyond the narrow scope of the FTC. But, he said, "We have been reviewing these matters and have identified some problems.

"We need to look at how the regulatory structure gives rise to greenmail," said Miller, one of the administration's most active proponents of deregulation.

He said objectionable defense tactics arise out of "a portfolio of restraints," such as federal requirements that reports be filed as stock is acquired and waiting periods between filing forms and buying stock.

The 1968 Williams Act, which governs the procedures in the takeover of a publicly held company, is a "potpourri of restraints," Miller said. Various provisions of the act, together with "various interpretations" by the Securities and Exchange Commission, create incentives for corporate defense tactics that cause public concern, he said. "Greenmail is a manifestation of an institutional arrangement that we might take a look at."

He also suggested reviewing the tax code to determine if it creates incentives for takeovers.

The current wave of takeover activity does not reflect any weakness in the antitrust laws or administration enforcement policies, the FTC chairman said.

Miller also said he opposes Commerce Secretary Malcolm Baldrige's proposal to repeal part of the antitrust laws. Baldrige recently proposed repealing Section 7 of the Clayton Act to improve U.S. competition in world trade by allowing more mergers that would improve economic efficiencies.

"The antitrust laws serve us well," Miller said, adding that the courts and federal agencies evaluate proposed mergers by analyzing foreign as well as domestic competition, and by considering the possibility of improved efficiencies.