Overlooking the Atlantic’s azure waters along the coast of Palm Beach, Fla., a seemingly endless line of megamansions hide behind tall walls. There, some of the planet’s richest people play host to glamorous balls and parties. But in one of those houses, tucked away on El Brillo Way and once guarded by winged gargoyles, such glamour collided with scandal.

The estate belonged to New York financier Jeffrey Epstein — a sex offender once linked with former president Bill Clinton, Nobel Prize-winning scientists, Kevin Spacey and British royalty. A stream of young girls allegedly flitted in and out of the house in the mid-aughts, attending naked pool parties and, police records showed, dispensing massages to Epstein and other guests.

Prince Andrew was allegedly one of the house’s visitors. On Friday, the Duke of York was named in a federal lawsuit filed against Epstein, whom the FBI once reportedly linked to 40 young women. Filed in 2008 in the Southern District of Florida, the $50 million lawsuit claimed Epstein had a “sexual preference and obsession for underage minor girls … gained access to primarily economically disadvantaged minor girls in his home, sexually assaulted these girls.”

A new motion now claims Epstein forced a young girl — referred to as “Jane Doe #3” — to have sex with Prince Andrew in London, New York and on a Caribbean island in “an orgy with numerous other under-aged girls.” According to the document, “Epstein instructed Jane Doe #3 that she was to give the Prince whatever he demanded and required Jane Doe #3 to report back to him on the details of the sexual abuse.”  The accuser said she was Epstein’s “sex slave” between 1999 and 2002.

Buckingham Palace denied the allegations. “This relates to long-running and ongoing civil proceedings in the United to which the Duke of York is not a party,” a palace statement said. “As such we would not comment in detail. However, for the avoidance of doubt, any suggestion of impropriety with underage minors is categorically untrue.”

This isn’t the first time Prince Andrew has had to bat away allegations about his relationship with Epstein — a disgraced businessman who pleaded guilty to procuring a person under the age of 18 for prostitution in 2008 and later served 13 months in prison. After his release in 2011, Andrew strolled through Central Park with Epstein. The news of a prince spending time with a registered sex offender quickly made news from New York to the United Kingdom. “What was the Duke of York thinking?” asked the Daily Telegraph.

The two men — one a garrulous New Yorker, the other a globetrotting royal — couldn’t appear more different. Raised in Coney Island, Epstein studied math but never got his bachelor’s degree. He wound up teaching calculus and physics at New York’s Dalton School, where his passion for teaching and mathematics caught the attention of one Wall Street father. “What are you doing teaching math at Dalton?” the father said, according to a New York magazine profile from more than a decade ago, “International Moneyman of Mystery.” “You should be working on Wall Street.” He soon  got a job at Bear Sterns, eventually opening up his own firm.

But Epstein remained an enigma to many. There weren’t any published photographs of him. For years, he didn’t grant a formal interview. And where did all of his money come from? “He is this mysterious, Gatsbyesque figure,” one “prominent Wall Streeter” told the magazine. “He likes people to think that he is very rich, and he cultivates this air of aloofness. The whole thing is weird.”

Donald Trump was Epstein’s pal at that time. “He’s a lot of fun to be with,” Trump said. “It is even said that he likes beautiful women as much as I do, and many of them are on the younger side.”

Vanity Fair found the same: “Epstein is known about town as a man who loves women — lots of them, mostly young.”

But even as those rumors swirled, Epstein nonetheless collected a staggering assortment of property — and high-powered acquaintances. He flew on planes with Spacey and Clinton in 2002. Through a spokesman, Clinton called Epstein a “committed philanthropist” at the time. He bought homes in Manhattan, New Mexico, Palm Beach and the island of Little St. James in the Caribbean, Vanity Fair reported.

One of the famous friends who entered Epstein’s fast-moving orbit was Prince Andrew, who appeared at a Florida function in 2000 with Epstein under the name “Andrew York,” the Telegraph reported. Soon, he was spending time at Epstein’s Palm Beach mansion, though Buckingham Palace denied he ever spent more than a few days there.

In 2008, the lawsuit came. According to reports in the Telegraph and Vanity Fair, one girl claimed she had been “sexually exploited by [Epstein’s] male peers, including royalty, politicians, academics, businessmen and other professional or personal acquaintances.” Prince Andrew repeatedly and forcefully denied he had any sexual contact with any girls at Epstein’s properties.

But Vanity Fair dug through sworn depositions around that time and found Andrew’s name.

“According to a sworn deposition by Juan Alessi, a former employee at Epstein’s Palm Beach estate, Andrew attended naked pool parties and was treated to massages by a harem of adolescent girls,” the magazine claimed. “At least three of the girls were questioned under oath about whether Andrew had had sexual contact with any of the masseuses.”

Alessi said Epstein’s parties got out of control. “European girls particularly, they were always taking their clothes off,” he said, the Telegraph reported. “I’d have to tell them, ‘Look, go away, put something on, and then come back here.'”

As for Andrew, Alessi said he remembered nothing but a “gentleman” of a man, who was “very ordinary, very easy-going. … He was probably the best guest we ever had and we had thousands. He would get up and make his own bed, tidy things up. … He was the only guy who left us a tip — and a key chain from the royal palace as souvenir.”