The Senate voted once again yesterday to lift the pay cap for high-ranking federal employes as Republican leaders scrambled to avert a presidential veto of a catch-all spending bill to keep the government operating next week.

Voting 54 to 41, the Senate approved pay raises that would average 4.8 percent for 46,000 government executives whose pay has been frozen since 1979, and is now at $50,112.

Some would get more than 4.8 percent, some less under the formula, but it was not clear any raises would finally pass. A similar Senate proposal died in a House-Senate conference less than two months ago.

The likely outcome of the intensifying struggle between President Reagan and Congress over what remains of Reagan's "September offensive" for budget cuts was also uncertain, although GOP leaders claimed early today they probably had enough votes to win Senate approval of the spending cuts Reagan needs to sign the bill.

Working against a deadline of 12:01 a.m. tomorrow, Senate Republican leaders agreed on a formula for cutting 4 percent off the big "continuing resolution" for funding the government; that would give Reagan at least half the domestic appropriations cuts that he wants for this year.

But, as the leaders worked past 3 a.m. today, they were still redrafting the measure in hopes of picking up enough votes for passage in the Republican-controlled Senate. Leaders of both parties in the Democratic House were saying a compromise there was possible if the Senate could produce a satisfactory plan.

Failure on the part of the Senate to include the proposed cuts would virtually assure a presidential veto because the administration claims the House-passed version of the measure fails, despite Democratic claims to the contrary, to meet Reagan's targets for spending cuts.

A weekend-long standoff between the White House and Congress could leave nearly all government agencies without funds to open for business Monday because current stopgap spending authority will expire at midnight Friday and Congress has not passed any regular appropriations bills except the one for its own operations.

The Senate was prepared to work all night, if necessary, in hopes of wrapping up congressional action and resolving any remaining disputes with Reagan today. White House officials said Reagan will not leave for his Thanksgiving vacation in California until a "satisfactory continuing resolution" is worked out.

As of early yesterday, Senate Republican leadership sources were claiming enough votes to pass the spending cuts package, but the votes seemed to vanish as the day wore on and the proposal was committed to writing. Leadership aides said it might not even be offered if failure seemed possible, noting that failure now could weaken Reagan's case for cuts after a veto.

However, after reworking the measure five or six times, they said shortly before midnight that an agreement was "within reach."

The Senate version of the resolution gives Reagan $2.5 billion worth of the $8.4 billion in cuts that he proposed last September in seeking a 12 percent reduction from spending levels for domestic appropriations that he had recommended in his original budget last March.

The new proposal, drafted by Sens. Pete V. Domenici (R-N.M.) and James A. McClure (R-Idaho) and presented by Senate Majority Leader Howard H. Baker Jr. (Tenn.) would cut another $3.2 billion, for an overall total of $5.7 billion.

By contrast, the House version cuts less than $1 billion, according to Reagan's Office of Management and Budget, although House Democrats contend that it cuts much more.

The new $3.2 billion in cuts would come from imposing a net reduction of 4 percent for all government activities except defense, military construction, foreign aid, the judiciary, veterans health benefits, basic benefit or entitlement programs, food stamps and revenue sharing with state and local governments, meaning it would apply to most major domestic appropriations. Later revisions also included an exemption for programs that already met Reagan's targets for cuts.

Veterans health benefits were exempted toward the end of the protracted negotiations, causing the total savings to drop from $3.6 billion to $3.2 billion.

In a final revision before bringing the revised draft to the floor, Senate leaders agreed to terminate the stopgap spending authority under the resolution March 31 instead of continuing it through the end of the fiscal year Sept. 30, as the original draft of the resolution provided.

This was done largely to satisfy senators who are unhappy with spending levels in the resolution and want to pass individual appropriations bills setting new levels. Spending levels in the resolution are based largely on House-Senate conference agreements on pending appropriations bills and on the lowest of the House and Senate versions of the bill.

They also agreed to bring up the defense appropriations bill Nov. 30 so the Senate can influence, and probably raise, the House-proposed Pentagon spending levels in the continuing resolution.

Reagan would have flexibility to cut up to 5 percent in any individual programs as long as the overall cut did not exceed 4 percent, and no program could actually be terminated.

The Senate proposal is roughly similar to one advanced by House Minority Leader Robert H. Michel (R-Ill.) in the House earlier in the week. It was rejected, 201 to 189, but the vote was close enough that Democrats as well as Republicans say a compromise might be worked out.

However, sources said Senate nose counts last night showed widespread Democratic opposition, with some Republican defections among conservatives as well as liberals. In addition to resistance to the cuts, they cited opposition to the exclusion of defense and entitlement programs and reluctance to give up congressional control over how program cuts are allotted.

As the Senate plowed through a long list of amendments, it rejected, 73 to 23, a proposal by Sen. Harry F. Byrd Jr. (Ind.-Va.) that would have established Reagan's budget as a spending level for the continuing resolution.

Approval of the pay raise for federal executives came amid warnings of a serious "brain drain" of top government officials. Senate Majority Whip Ted Stevens (R-Alaska) who made the proposal, said there is now little incentive for executives to stay in the federal system when they can earn more in private industry.

Stevens' proposal would raise pay for executive levels three, four and five from their current frozen rate of $50,112 to $59,360, $58,500 and $57,500, respectively.

Stevens introduced a separate amendment giving members of Congress a 4.8 percent pay raise, which failed, 90 to 5.

The only Washington-area senator to vote against lifting the federal pay cap was Byrd.

Meanwhile, the Senate Budget Committee grudgingly abandoned its drive to come up with long-term spending and tax increases to balance the budget by 1984 and sent to the Senate floor a "stopgap" budget resolution that acknowledges the likelihood of very large deficits over the next three years.

The resolution, which the committee said it was passing along "reluctantly" and "without recommendation," merely rubber-stamps the budget targets approved last spring--without accounting for bad economic news that in the meantime has driven up budget projections. The committee explained that it still favors a balanced budget by 1984 but could not agree on a means to that end.

As a result, the committee noted "current budget projections" for deficits in the range of $76 billion to $92 billion in 1982, $95 billion to $136 billion in 1983 and $103 billion to $165 billion in 1984.