United States Attorney for the Southern District of New York Geoffrey Berman speaks during a news conference, in New York last week. Federal prosecutors announced sex trafficking and conspiracy charges against Jeffrey Epstein. (Richard Drew)

It’s not just his freedom that’s at stake. It’s his fortune.

The criminal prosecution of New York moneyman Jeffrey Epstein on charges of sex trafficking and the lawsuits stemming from it could dwindle his financial empire to nothing, attorneys with knowledge of the case said.

While much of the legal sparring that began with an indictment last week has focused on questions of Epstein’s freedom, a secondary struggle may determine how much money Epstein has — and what he will owe and to whom when it’s all over.

Whether Epstein is a billionaire, as he has been routinely described, is difficult to know. None of the known paper trail establishes a fortune of that magnitude. But by most accounts he has substantial wealth, and the legal assault upon it has begun.

During a hearing Monday in a New York federal courtroom, prosecutors and Epstein’s attorneys made arguments before U.S. District Judge Richard Berman about whether Epstein should be released pending trial and, if so, what the conditions of release should be. Berman said during the hearing he plans to make a ruling Thursday and that a pre-trial services report recommends Epstein be detained, according to multiple news reports.

Epstein resolved a similar investigation years ago by striking a plea deal with prosecutors under which he spent 13 months in jail. The deal has been criticized as too lenient and on Friday, the prosecutor who approved it, Alexander Acosta, announced he was stepping down as U.S. labor secretary.

In addition to seeking prison time for Epstein, federal prosecutors have signaled they will aim for the forfeiture of his $60 million New York home. Under federal law, properties such as his $13 million Palm Beach home, his properties in the Virgin Islands, his jets and a $12 million New Mexico ranch may be vulnerable to similar claims.

His defense attorneys and related expenses are likely to cost him tens of millions more.

And finally, lawsuits from the alleged victims — of which there are scores — could run over a billion, experts said. In all, these expenses could overtake his wealth.

“With aggressive lawyering for the victims, Mr. Epstein’s realistic financial exposure for criminal restitution in federal court is likely to exceed $500 million, and the exposure in state court to exceed a billion,” said Paul Cassell, a University of Utah law professor, a former federal judge and a lawyer for one of Epstein’s alleged victims. “It’s easy to foresee 160 victims in this case and possibly more.”

Prosecutors in New York reportedly have said there could be hundreds of victims around the country, and the Miami Herald says it has identified as many as 80 alleged victims. Some already have sued and entered legal settlements with Epstein. In announcing the recent indictment, the Justice Department said Epstein had “a vast network of underage victims."

Those victims may be entitled to collect money from Epstein for their economic losses, such as the expense of therapy, for their pain and suffering, and for punitive damages.

But it is his collection of real estate that is likely to be the first casualties in his legal war. Under federal statutes, prosecutors can seek the forfeiture of any properties where the sex trafficking may have taken place.

In order for the government to take a property in such cases, there has to be “some nexus between the property and the offense,” said Stefan Cassella a former federal prosecutor and expert on asset forfeiture and money-laundering law.

The indictment unsealed last week charges that the offenses occurred at Epstein’s New York and Palm Beach homes, suggesting that only those properties would be subject to forfeiture.

But if evidence shows that his jets or other residences were also involved, those also could be lost, Cassella said. The Santa Fe New Mexican reported last week that the New Mexico attorney general, Hector Balderas, is investigating whether Epstein committed any crimes in that state, where he owns a place called the Zorro Ranch.

Finally, beyond the financial penalties that could be exacted by federal prosecutors and the scores of alleged victims, are the substantial bills run up by his own legal team.

The team already is rich in celebrity. It has included Alan Dershowitz, the noted constitutional law professor, and Kenneth W. Starr, who investigated president Bill Clinton’s relationship with Monica Lewinsky.

“In a big time criminal prosecution, a defendant can easily spend millions and in some cases tens of millions of dollars through trial,” said Matthew L. Schwartz, a white collar defense attorney and a former federal prosecutor who worked on the Bernard Madoff case.

Against all those potential legal liabilities stands Epstein’s mysterious fortune, the magnitude of which is difficult to discern.

In fact, little is known about how Epstein, 66, the son of a Coney Island parks department worker, acquired his riches. While he is often described as a billionaire in media profiles, there is little in the public record, aside from his real estate, to substantiate the claim. He also has been reluctant to disclose what he owns.

In a 2013 civil court filing in Palm Beach County Circuit Court, Epstein responded to a question about his net worth without giving an exact figure. “I have already indicated my willingness to stipulate to a net worth in excess of one hundred million dollars,” he wrote.

In the same court filing, he described himself as “self-employed and philanthropist,” but said he could not provide answers “to questions relating to my financial history and condition” without waiving his constitutional rights.

“I don’t think he’s a billionaire,” Thomas Volscho, a sociology professor at the College of Staten Island — CUNY, who has been researching a book about Epstein, said last week. “His wealth is possibly above $200 million.”

Indeed according to a court document filed by Epstein and unsealed on Monday, he has assets worth $559 million. Most of that wealth - about $378 million - is kept in hedge funds, private equity funds and other investments. The remainder consists of real estate, including his properties in the U.S. Virgin Islands, which are worth $86 million, according to the document.

Whatever the outcomes of these investigations, the long-standing riddles posed by Epstein’s fortune could be answered in the near future, as various parties who believe Epstein is in their debt — the federal government, alleged victims and defense attorneys — seek to collect.

“Hiding money is difficult,” said Stephen Harbeck, former president and chief executive of Securities Investor Protection, a federally chartered agency in Washington that safeguards brokerage accounts. “You can trace where the money came in and where it went out. Money goes through banks, and banks are required to make extraordinarily careful records.”

Renae Merle contributed to this story.