After cracking down on rapid crime growth by jailing more criminals and imposing longer sentences, Florida is faced with the unhappy choice of building more prisons or letting criminals back onto the streets.

A federal court order to end overcrowding in the state's 26 major prisons and 50 smaller penal centers has created what Attorney General Jim Smith calls an "immediate crisis," requiring construction of new cells for 4,000 convicts within two years or early release of prisoners left without beds.

To complicate matters, state legislators are reluctant to spend more money on jails, and local authorities are fighting against new prisons in their areas.

State governments are increasingly confronting such crises as courts hand down longer sentences. Prisons are filling up at the fastest rate since the Justice Department began keeping count in 1926, reaching a national total of more than 400,000.

Prison systems in at least 10 states, the District of Columbia and Puerto Rico, are under court orders similar to that in Florida. Individual prisons in 26 other states also have been ruled unfit.

Chief Justice Warren E. Burger warned last month in his annual report that the spread of mandatory sentencing and tough anti-crime laws "may well enlarge the prison population and lead to more prison explosions."

Florida, beset by drug smuggling and illegal immigration, is an extreme case. It has the nation's fourth largest prison population--27,000--behind Texas, California and New York. Its 1982 increase--about 4,000--was second only to California, forcing the housing of 1,200 prisoners in prefab cells.

Six months ago, Gov. Robert Graham named a task force on prison overcrowding to recommend legislation to meet the crisis. It is nearing the end of its work and is undecided what to do, despite a warning from Smith that new construction should begin by next summer.

"We are facing in the next 24-30 months the prospect of having several thousand people who aren't fit to be in a free society being released in a free society," Graham told a recent task force meeting.

His warning referred to recommendations from task force members that the state have an "emergency release mechanism" in place by fall. The plan would take effect automatically if prison population rose more than 5 percent over the current ratio of 240 convicts per 100,000 residents.

It would accelerate parole and, if necessary, provide for temporary "house arrest" of nonviolent overflow prisoners. Under special circumstances, it would also allow early release of inmates serving mandatory 25-year terms for capital crimes.

In a similar action, the state parole commission has requested authorization to take prison overcrowding into account in deciding on paroles.

Whatever happens to such proposals, state authorities and task force members recognize that new prisons must be built. But building a jail is no simple matter, they are learning. State Sen. Edgar Dunn, a task force member, has suggested, for example, that prison planning be coordinated with other regional planning to avoid conflicts and make sure related issues such as access roads and guard housing are taken care of. Graham lauded Dunn for farsightedness, but added: "We have an immediate problem that can't wait for other things to move."

Aside from concern about appropriating tax money, Florida's foremost obstacle to new prisons appears to be Floridians themselves. Pressured by voters, local governments have mounted strong resistance to state efforts to locate new penitentiaries in their backyards.

"The real world is that, politically, no one wants to vote zoning for prison sites," Smith said.