Secretary of State Edmund S. Muskie will visit Venezulela, Peru, Brazil and Mexico immediately after the U.S. presidential election, the State Department announced yesterday.

Muskie's trip, his first to Latin America since taking office last May, was described by a department spokesman as "reflecting the importance we place on close, cooperative relations with our neighbors in this hemisphere."

The spokesman added that the visit will take place even if President Carter is defeated on Nov. 4 and Muskie finds himself a lame duck in the secretary's office.

Another potential cloud over the trip involves the uncertainty about whether the American hostages in Iran will be released soon. Asked what would happen if that question is still unresolved on Nov. 6, when Muskie is to begin his trip, spokesman John Trattner said, "We'll have to think about it in respect to what the situation is at the time."

The visit, which department officials said was planned before the hostage situation began showing possible movement, is intended to signal Muskie's intention, after having his attention long diverted elsewhere, to start involving himself in the problems of a region that has complained of neglect by the Carter administration.

One official described the plan as "almost a total immersion course in Latin American affairs" for the secretary. Almost immediately following the Nov. 6-to-13 trip, Muskie will head the U.S. delegation to the annual meeting of the Organization of American States, beginning here Nov. 19.

Each of the tour countries was chosen with a specific purpose.

Venezulela is Latin America's major oil producer, and Mexico, in addition to being this country's next-door neighbor, is expected to become one of the world's largest oil producers. According to U.S. officials, Muskie's visit to Peru is intended to demonstrate support for the reestablishment of democracy last August after 12 years of military rule. Brazil, unlike the other three, is under a military dictatorship, but it also is the largest and most powerful country in Latin America.

For these reasons, the Muskie trip could be very significant in terms of long-range U.S. relations with Latin America, provided the Carter administration remains in office. However, if the Republicans win the election, Muskie's visit will take on little more than a ceremonial character since Ronald Reagan is expected to reverse many of President Carter's Latin America policies, such as a get-tough approach to many hemispheric dictatorships and an effort to cooperate with radical leftist forces in Central America and the Caribbean.

Muskie's itinerary calls for him to visit Carcas, Venezuela, Nov. 6-7; Lima, Peru, Nov. 7-9; Brasilia Nov. 9-11, and Mexico City Nov. 11-13.