Chubby Florida manatees are adorable.

Lumbering in the cozy waters of their habitat, they look like big, soft, squishy gray pillows. Anyone who wants to get in and swim alongside them is in luck at the Crystal River National Wildlife Refuge on the central gulf coast of Florida. U.S. Fish and Wildlife Service officials who manage the refuge say go ahead, they don’t bite.

But whether that’s good for the beloved “sea cow” is a question that could one day be resolved by a threatened lawsuit. A group called Public Employees for Environmental Responsibility is deeply upset that the refuge allows swimming with an endangered marine mammal in the warm springs that serve as their winter sanctuary.

People are loving the manatees to death, the group says, and Fish and Wildlife has let the smothering affection develop into a lucrative tourism industry at Three Sisters Springs in Citrus County. PEER filed a notice of intent to sue Fish and Wildlife if they don’t tell tourists to back off. Fish and Wildlife has until May to respond.

“Swim with programs significantly impair these endangered animals’ essential behavioral patterns, including breeding, feeding, and sheltering,” said the group’s counsel, Laura Dumais. “Some people have a hard time understanding this connection, because they don’t see manatees keeling over before their eyes; they might think that the manatees don’t seem to mind.”

The manatees certainly don’t seem to mind, the refuge’s manager said. Florida’s manatee population topped a record 6,000 this week in the state’s latest survey, nearly 1,000 more than the previous high. The numbers are so high that some are calling on the service to remove the manatee’s protection under the Endangered Species Act, where they’ve been listed since it began in 1973. A review that might downgrade them from endangered to threatened is in the draft stage.

“The manatee is actually a success story. Their numbers are going up, the population is going up,” said Andrew Gude, manager of the refuge. “Tourism has also gone through the roof. You can rent a car and for $40 you can swim with a mammal that will never rip you apart. The reason the service has been so supportive is that when people see the manatees and get in the water with them, in a lot of ways it changes their lives and they’re a lot more conservation-minded.”

This time of year, when cold-sensitive manatees migrate to the warm springs because water temperatures below 68 degrees could kill them, hundreds of thousands of tourists rush to where they congregate to gawk at and touch them. They snorkel, kayak, raft, scuba, boat and swim free style.

With 327,000 visitors last year, the Crystal River refuge was the 5th most visited in the nation, with sweet, lovable manatees as the main attraction. As hundreds of thousands of humans thrash in the water for a moment of intimacy and, of course, a photo op, the narrow swimming channels the manatees use to come and go are blocked, and studies show that some stay away, not wanting to be bothered by the commotion, PEER said. A concern is that manatees will risk deadly cold gulf waters.

“It is the behavior that doesn’t happen that’s problematic – the manatees that see swimmers crowding the run and don’t enter the spring,” Dumais said.

In a recently adopted assessment to manage manatees, Crystal River didn’t decide to ban all human contact with manatees as PEER wants, but the refuge does take additional steps to protect them. It will close the sanctuary to tourism during extreme cold, when manatees need it most. And it expand an area where humans are prohibited, allowing more manatees to avoid being touched.

Manatee numbers are rebounding now, she said, but history shows that the population of this sensitive creature could take another dive at any time. Nearly 800 were killed in 2010, and an extended cold snap was blamed for 300 of those deaths. Three years later, there were a record 800 deaths. Fish and Wildlife estimates that 99 manatee deaths per year are related to humans. The population’s low in Florida was about 1,400.

Manatees have been in the state for 45 million years, according to fossil records. They are an offshoot of the West Indian manatee that roams the Caribbean Sea and the Atlantic Ocean to north Brazil, and the Gulf of Mexico from southern Mexico to Colombia. In those areas, they are scattered in much smaller numbers.

They are totally chill. Not known to harm anything, they spend their days diving to dine on sea grasses and fresh water vegetation. But humans harm them with watercraft collisions and boat propellers that slice their skin. Mortality is so common that the state Fish and Wildlife Conservation Commission has a phone line to report dead manatees.

Floridians call them sea cows, and they have marveled at them and swum with them at the popular Three Sisters Springs at the refuge in Citrus County for decades. In time, tourists from outside the state caught on.

Why allow all those people to swim with manatees and boat around them for just a few bucks? “The boardwalk at Three Sisters Springs … provides an unparalleled opportunity to view manatees in their natural habitat,” Dumais said.

It’s not like manatees are being harmed, said Charles Underwood, public information officer for Fish and Wildlife’s north Florida office, which is in charge of manatee recovery. When they are, even the smallest infraction is dealt with. “We do prosecute any harm to manatees,” he said. Like a group of kids who recently lured a manatees to them with cabbage “and did a cannonball on them. It’s a violation. It was a significant form of harassment.”

Read more: