U.S. Environmental Protection Agency Administrator Lee M. Thomas, reacting to growing concern over factory, military base and sewer plant discharges into the Chesapeake Bay, called yesterday for dramatic reductions in the amount of pollutants allowed in those discharges.

Dischargers currently operate under state permits that set dumping limits according to what cleanup technologies are available and what dischargers say they can afford to do. A number of toxic pollutants are not limited; other pollutants are restricted, but the restrictions allow large amounts of the pollutants to be dumped into the bay.

Thomas told a Senate subcommittee headed by Sen. Charles McC. Mathias Jr. (R-Md.) that the bay states and the District should begin basing permits not on what dischargers can do but on what would improve the quality of water in the bay.

He said that a water-quality-based permit system is being followed by states adjoining the Great Lakes. State and D.C. governments should revise along the lines of the Great Lakes model a bay cleanup agreement they wrote in 1983, Thomas said. EPA officials said later that the agency will negotiate such revisions with state governments in the next year.

Thomas' comments follow a Washington Post story this month on the decline of the Chesapeake, the nation's largest bay, which showed that part of the reason for the decline is the rudimentary limits set out in Maryland and Virginia permits. The article said that dischargers routinely violate permits and are not being punished by state enforcers.

Thomas said yesterday that the story prompted an EPA audit of bay pollution and that enforcement of the permit system needs to be more aggressive.

Under questioning by Mathias, who is regarded as the father of the multimillion-dollar Chesapeake cleanup accord, Govs. Harry Hughes of Maryland and Gerald L. Baliles of Virginia balked at the need to forge a new cleanup agreement.

Hughes said that new and tougher limits will come about eventually under the current accord.

The governors and D.C. Mayor Marion Barry said that in order to continue their cleanup campaign they will need more than the $10 million a year that has been set aside by the federal government until 1988.

"No plan, however well executed, will suffice without adequate funds to support it," Baliles said. Hughes said that Maryland, which has spent $130 million to clean up the bay, needs $60 million from the federal government over the next five years to reduce the amounts of toxic chemicals and nutrients being spewed into the bay.

In calling for stricter limits on pollutants, Thomas cited the successful eight-year Great Lakes Water Quality Agreement between the United States and Canada. That program allows industries with permits to discharge only minute amounts of 16 pesticides, 10 heavy metals and a variety of other toxic chemicals. It also limits the amounts of nitrogen that sewage treatment plants are allowed to discharge into the Great lakes.

Environmentalists William Baker, president of the Chesapeake Bay Foundation, and Charles Fox of the Environmental Policy Institute said that a more detailed bay cleanup plan -- including water quality standards and a cap on the amounts of toxic chemicals and nutrients such as nitrogen -- is long overdue.

"While I'm not going to say it's a total smashing success, they Great Lakes dischargers are meeting those permit levels," said Fox.