The Agriculture Department is withholding at least $10 million in food aid from pregnant women and infants despite a congressional mandate that the money be reallocated so clients in some states won't be dropped from the program, a suit filed yesterday alleges.

The suit, filed by the Food Research and Action Center (FRAC) and three members of Congress, contends that nearly 50,000 women and children in New York and Georgia are being deprived of supplemental food under the Women, Infants and Children (WIC) program because extra funds have not been reallocated from states that have fewer clients than expected.

Reps. Wyche Fowler Jr. (D-Ga.), Matthew F. McHugh (D-N.Y.) and Ted Weiss (D-N.Y.) joined in filing the suit.

FRAC Director Nancy Amidei said Secretary of Agriculture John R. Block's action "shows a cavalier disregard of the Congress, the taxpayers who pay his salary and thousands of nutritionally vulnerable women and babies." Agriculture officials could not be reached for comment.

The Reagan administration has tried to cut funding for the WIC program by a third and put it into a block grant. But Congress has continued to appropriate nearly $1 billion a year for the program, which serves 2.2 million women and children.

Last year, the suit charges, nearly 250,000 WIC clients lost benefits because their states ran out of money, although the department ended up with more than $50 million in unspent funds. When this year's WIC appropriation was approved, Congress made clear it did not want a repeat, the suit said, citing conference report language.

Amidei said in a telephone interview that the Agriculture Department made one reallocation in April, but refused to make another last month, as it had promised. New York and Georgia have requested additional funds, but have been denied them, despite the surplus in some other states and the congressional mandate, she said.

The average WIC client allocation is $28 per month.