Former CIA director William E. Colby normally would not be considered a favorite of feminist activists, but he drew a rousing ovation this week from several hundred women gathered at Georgetown University.

"I'm an unabashed supporter of the nuclear freeze," Colby told the women, who were attending a three-day leadership conference on U.S.-Soviet relations. "I think it's time people take this matter away from the priesthood that has gotten us into this mess, and simply insist that we stop building these things. Just stop."

The women couldn't have agreed more. To many of them, world peace and the nuclear freeze are priority issues.

Coming from Colby, CIA director during the Nixon and Ford administrations, the words were particularly comforting to many in the audience.

"He speaks as a very secure man who is not threatened by the 'macho man' complex so many men have about the nuclear issue," said Betty Bumpers, leader of a freeze group called Peace Links.

The three-day conference, which ended yesterday, was sponsored by the Committee for National Security, composed primarily of liberal Democrats. Most of the 300 women who attended are political moderates, active in groups such as the PTA, the League of Women Voters, the American Association of University Women and a host of anti-nuclear groups.

Among those in attendance were about 10 congressional wives, including Bumpers, whose husband is Sen. Dale Bumpers (D-Ark.), and Teresa Heinz, whose husband is Sen. John Heinz (R-Pa.).

Conference leaders said women will play a decisive role in the future of nuclear arms control and U.S.-Soviet relations.

"If we don't have some creative new thinking, the future is very bleak for all of us," said Bumpers. "That's why women are so important. They don't buy what our leaders are telling us. They want our leaders to come up with alternatives to war because they don't want their children to be the last generation on this planet."

Mary Dent Crisp, a former co-chairman of the Republican National Committee and now an outspoken critic of New Right conservatives, said other issues have "begun to pale" when compared with the question of nuclear weapons.

"To me, if I don't make a difference on this issue we won't be around for anything else," she said.

The conference, financed in part by the Rockefeller and Ford foundations, attracted women from 38 states. Anne H. Cahn, CNS director, said the idea was to expose women leaders to a variety of opinions about U.S.-Soviet relations.

"We think our policies would be different if more women were involved in them," she said. "I'm not saying all wars would end, but women are less committed to defending the status quo, and are more open to exploring new ways of doing things."

The majority of the speakers were men. They included 12 officials from the Reagan administration; two from the Institute of the USA and Canada, an unofficial, Moscow-based group of Americanologists, and several Carter administration officials such as Paul H. Warnke, a former director of the Arms Control and Disarmament Agency.