The Senate fattened its version of the federal budget again yesterday with a $2.3 billion increase in education spending over the next three years, while frustrated Republican leaders introduced their long-promised fiscal 1984 budget compromise without having enough Republican votes to pass it.

In a measure of the uncertainty hanging over the budget as a final vote approaches next week, Republicans began to scout Democratic ranks for support for their substitute, and Democrats put out feelers to dissident Republicans for a possible compromise of their own.

With Republican maneuvering ability constrained by sharply opposing views on taxes in the party's left and right wings, the Democrats attempted to seize the offensive by offering flexbility, especially on taxes.

In what appeared to be a signal to Republicans, Sen. Lawton Chiles (D-Fla.), ranking Democrat on the Budget Committee, indicated that he is not irrevocably wedded to the proposed fiscal 1984 tax increase of $30 billion in the Senate committee's proposed budget, which he said could be used as a "starting point" in any negotiations.

Chiles reiterated that he never thought Congress would succeed in repealing the 10 percent July income tax cut, which Reagan said he is determined to preserve, and he suggested other revenue sources, such as excise tax increases or a surtax, to meet whatever tax target Congress sets.

Other sources indicated that the Democrats may seek enough Republican allies to defeat the GOP leaders' substitute as a prelude to development of what they call a "real" bipartisan budget.

As introduced by Budget Committee Chairman Pete V. Domenici (R-N.M.), the Republican leadership's compromise would drop the committee's demand for more than $120 billion in new taxes through 1986, while going along with its proposed increases in domestic spending and splitting the difference between Reagan and the committee on defense.

Although neither side claimed to have won over any defectors from across the aisle, the Democrats, in the only major vote yesterday, picked up enough Republican support to force a substantial increase in education spending and nudge the total level of domestic spending even farther beyond what President Reagan wants.

This added $250 million to projected spending for next year. Combined with Budget Committee proposals and the Senate's decision Thursday to add $900 million for health insurance for the unemployed, it would give Reagan $12.6 billion more than he wants for domestic spending in fiscal 1984.

The Senate on Thursday had approved a smaller $1 billion three-year boost in education spending, in an attempt to head off an even larger increase, adding up to a three-year total of $3.6 billion, that was proposed by a bipartisan group led by Sens. Ernest F. Hollings (D-S.C.) and Robert T. Stafford (R-Vt.).

But Hollings came back again yesterday with the $2.3 billion proposal, and it was approved, 55 to 32, with 17 Republicans joining most Democrats in voting for the increase. Many Republicans who voted for it are up for reelection in 1984.

Although yesterday's action represented a compromise, Republican leaders fought hard to defeat it, even asking Vice President Bush to make a rare appearance to cast the deciding vote in case of a tie.

As it turned out, although one key procedural maneuver was decided by a 2-vote margin, Bush's vote was not needed.

Domenici's substitute budget would produce a deficit of $192.4 billion next year, roughly equal to Reagan's, and considerably higher than the deficits projected by either the Senate Budget Committee or the Democratic-controlled House in the budget it approved last month.