Senate Majority Leader Robert C. Byrd (D-W. Va.) yesterday reemphasized his support for a balanced federal budget in the coming fiscal year and criticized those talking already of abandoning the plan.

"I'm a little amazed at the Nervous Nellies who want to jump ship," Byrd told reporters at his regular Saturday news conference. "We ought not to talk ourselves out a balanced budget so soon."

Though he wouldn't mention either by name, Byrd seemed to be criticizing President Carter and Sen. Edward M. Kennedy (D-Mass.), who has yet to give up his fight for the Democratic presidential nomination.

Kennedy told a meeting of the liberal Americans for Democratic Action here yesterday that it was irresponsible to talk about balancing the budget during a recession.

"When the real world of recession is wrecking our economy it is not responsible Democratic action for a Democratic Congress and a Democratic administration to act as though the budget could still be balanced," he said.

"We can reach the goal of a budget balance in good and proper time. But not in 1981 and not by pursuing phantom policies that ignore the fiscal reality pressing in on us."

Administration budget experts concede privately that growing unemployment will reduce tax revenues for the government, increasing the cost of unemployment benefits. Kennedy has called for a $12 billion antirecession program as a cure. And Carter has begun talking about the possibility of a tax cut to aid recovery from the recession.

Byrd said yesterday that he thought it was inadvisable to be talking about a tax cut so soon. "Any sizable tax cut would restimulate the fires of inflation," he said.

Byrd maintained that the prime factor in the recent drop in interest rates has been the signal from Congress that it is determined to exercise fiscal restraint. "This could be a self-correcting economy," he said. "The dip could be short-lived. . . We must not overreact."

Byrd also went out of his way to take a jab at probable Republican presidential nominee Ronald Reagan. He said a string of events in the past week has been "a sneak preview" of a Reagan foreign policy.

"He appears to be advocating a rerecognition of Taiwan. I think this would throw a chill on one of the most important foreign policy initiatives" in recent years -- the normalization of relations with the People's Republic of China.