More than half the scientists responding to a government survey on the use of aspartame, the enormously popular sugar substitute marketed as NutraSweet, voiced concerns over its safety, according to a Government Accounting Office report released yesterday.

Contained in dozens of products, including diet sodas, breakfast cereals and vitamins, the artificial sweetener is consumed by 100 million Americans each day. Even before it was approved for use by the Food and Drug Administration in 1981, the product was controversial. Critics charge that it might cause brain tumors, seizures, mood swings and headaches.

Because it has no medical expertise, the GAO made no recommendations in its report. It found that the FDA adhered to proper procedures by approving the sweetener. But the investigators also said "we cannot comment on whether important issues surrounding aspartame's safety remain unresolved."

Officials of the NutraSweet Company hailed the results, and so did the sweetener's strongest critics.

"The report provides additional evidence to add to the overwhelming consensus among scientists and health officials . . . that aspartame is safe and that the process through which it was approved was proper and adequate," the company said in a statement.

The company dismissed concerns voiced by scientists who were polled that the substance should have been put through more rigorous tests, in part because some of those questioned have not conducted research on aspartame.

The FDA has received more than 3,000 complaints about the powerfully concentrated artificial sweeteener since it appeared on the market six years ago. Consumers complained of dizziness, headaches and nausea after eating or drinking products with NutraSweet in them.

"They never asked the right questions about what it does to brain function in humans," said Dr. Louis Elsas, director of medical genetics at Emory University Medical School. "They decided without data that you had to have enormous amounts of phenylalanine in your blood before it becomes a problem. We don't know that's the case."

Phenylalanine, an amino acid, is the principal ingredient in NutraSweet. Those who suffer from phenylketonuria (PKU), a genetic disorder, cannot process the amino acid properly. All products containing NutraSweet must warn against its use where PKU exists.

Elsas and other researchers say they believe that aspartame can do more damage over a long period of time than federal health officials have said they think.

FDA officials have said repeatedly that aspartame has been subjected to stricter tests than any product ever approved. They declined comment yesterday, however, because they had not yet received a copy of the report.

The GAO report noted that the FDA Center for Drugs and Biologics, which oversees testing of new drugs, originally recommended that NutraSweet should have the more stringent testing required for drugs, rather than foods. FDA officials overruled the recommendation.

The report also said that before it was approved in 1981 by former FDA Commissioner Arthur Hull Hayes Jr., three of five FDA scientists advising the commissioner told him that studies conducted by the manufacturer had not conclusively proved the sweetener was safe.

The FDA polled 69 medical researchers, some studying aspartame. Twelve said they had "major" concerns about aspartame safety; 26 said they were "somewhat concerned," and 29 said they were "very confident" of the product's safety.