Norman Braman, President Reagan's nominee to head the Immigration and Naturalization Service, unexpectedly withdrew his name from consideration this week, citing business considerations.

Braman is a wealthy Miamian, who owns the nation's second-largest Cadillac dealership among others.

In a letter to Attorney General William French Smith, he said, "My business is the sale and service of automobiles. With the current depressed market affecting our industry, I have concluded that, notwithstanding the present profitability of my enterprises, I cannot now assume a passive role in their operation, which full-time public service would require."

Smith sent Braman's name to the White House in June, and it was formally sent to the Senate Sept. 29. Senate staff members who have been investigating Braman's background say they had not turned up any problems. But their investigation had not been completed, and some said they believe Braman may have grown tired of waiting.

Braman's nomination was criticized by a number of groups concerned with immigration policy. They charged that he was inexperienced in both government management and the complex problems created by the increasing number of aliens seeking entry into this country.

INS has been run for more than two years by acting commissioners, and with the flood of Cuban and Haitian immigrants into Florida, Reagan has been under pressure to name a permanent head, particularly since he sent Congress his proposals for changing the country's immigration policy.

Congressional sources said yesterday that the leading candidate for the job now is Alan C. Nelson, currently deputy commissioner under acting commissioner Doris Meissner. The sources said they believe Nelson, who has testified frequently before Congress, would be confirmed easily. Nelson was a member of Reagan's cabinet in California.