Louis Farrakhan, controversial Black Muslim minister, said yesterday he will visit Libya in defiance of President Reagan's ban on travel to that country.

"I personally plan to travel to Libya at some time during the year," Farrakhan said during a news conference at the National Press Club. He added that he would challenge the constitutionality of the restrictions Reagan has placed on travel and work by Americans in Libya.

Reagan imposed the economic sanctions and travel restrictions after the Dec. 27 airport attacks in Rome and Vienna, accusing Libya of supporting the Palestinian gunmen who threw grenades and sprayed submachine gunfire into crowds, killing 20.

"Libya is a base, not for terrorist activities, but Libya is a base for the support of all those who fight against tyranny and oppression in the world," Farrakhan said yesterday.

Farrakhan charged that Reagan is trying to provoke a war with Libya by recently sending two U.S. aircraft carriers back to waters off Libya and said he would tell black servicemen "not to fire a shot."

Farrakhan received a $5 million, interest-free loan from Libya during a trip last year that he said he would use to market personal care products for blacks.

"If it is against the law for me to pay back the loan that we borrowed from Libya, I will have to take my chances," he said.