Police in Rio don’t seem to believe Ryan Lochte’s robbery story. (Matt Hazlett/Getty Images)

A U.S. Olympic Committee spokesman said Wednesday that Brazilian police have attempted to collect the passports of 12-time Olympic medalist Ryan Lochte and U.S. swimming teammate Jimmy Feigen as they continue to investigate the swimmers’ claims they were robbed at gunpoint in Rio de Janeiro early Sunday.

“Local police arrived at the Olympic Village this [morning] and asked to meet with Ryan Lochte and James Feigen and collect their passports in order to secure further testimony from the athletes,” USOC spokesman Patrick Sandusky said Wednesday. “The swim team moved out of the village after their competition ended, so we were not able to make the athletes available.

“Additionally, as part of our standard security protocol, we do not make athlete travel plans public and therefore cannot confirm the athletes’ current location.

“We will continue to cooperate with Brazilian authorities.”

Lochte already has returned to the United States, his father confirmed to the Associated Press on Wednesday. According to Steve Lochte, his son called him Tuesday to say he was back and that he was going to “buy a new wallet to replace the one that he said was stolen from him in the robbery.”

Jeff Ostrow, Lochte’s attorney, told CNN on Wednesday that the swimmer would not be returning to Brazil to answer further questions and that he stands by the statements he gave to Rio police, the FBI, the State Department and the USOC’s security team earlier this week.

“We stand by what he provided in that interview and signed off on,” Ostrow said.

Feigen told the San Antonio-Express News that he was still in Brazil but declined further comment.

According to the Brazilian daily O Globo, a Brazilian judge issued an order banning the four U.S. swimmers involved in the incident from leaving the country. O Globo said Brazilian authorities are troubled by the contradictory statements given to them by swimmers Lochte, Feigen, Gunnar Bentz and Jack Conger, who claim they were on their way back to the Olympic Village from the French Olympic team’s hospitality house in a taxi when they were robbed at gunpoint. Police in Brazil have asked the court to issue a search warrant for an apartment used by the swimmers at the Olympic Village, specifically so they can obtain Feigen’s cellphone to track where the foursome was when the robbery occurred.

The Associated Press reported Tuesday night that Rio police have found little evidence to support Lochte’s robbery claim. According to an unnamed AP source with knowledge of the investigation, authorities cannot find the taxi driver or any witnesses. According to the BBC, Lochte and Feigen told police they returned to the Olympic Village at 4 a.m. local time, even though security camera footage appears to show them returning closer to 7 a.m.

Here’s what Lochte told NBC on Sunday, before he talked with authorities that night:

We got pulled over, in the taxi, and these guys came out with a badge, a police badge, no lights, no nothing just a police badge and they pulled us over,” Lochte said. “They pulled out their guns, they told the other swimmers to get down on the ground — they got down on the ground. I refused, I was like we didn’t do anything wrong, so — I’m not getting down on the ground.

“And then the guy pulled out his gun, he cocked it, put it to my forehead and he said, ‘Get down,’ and I put my hands up, I was like ‘whatever.’ He took our money, he took my wallet — he left my cellphone, he left my credentials.”

The unnamed Brazilian official told the AP that the group of swimmers did not immediately call the police, who only began investigating once they saw media reports in which Lochte’s mother spoke about the robbery. Lochte and one of the other swimmers — not named by the AP — told police that they were intoxicated at the time of the incident and could not describe the color of the taxi, the neighborhood where the incident occurred or what time it took place.

As part of their ongoing investigation, police are looking for security-camera footage from the French hospitality house and nearby gas stations. The AP source’s says security-camera footage obtained from the Olympic Village shows the swimmers returning at 6:56 a.m. on Sunday morning.

It should be noted, as the AP does, that police in Rio often struggle to solve robberies in the crime-ridden city, given their lack of resources. Many robberies aren’t even reported.

On Tuesday, Lochte told USA Today’s Nicole Auerbach that he did not inform U.S. Olympic Committee authorities about the incident for a number of hours Sunday “because we were afraid we’d get in trouble.” It’s unclear why Lochte would think he would be getting in trouble. Under IOC rules, athletes are allowed to consume alcohol in private spaces of the Olympic Village.

Fake-police robberies like the one Lochte described to NBC are common in Rio though not, the AP reports, in the area where Lochte’s group allegedly was robbed.