The Senate plunged into a full-scale filibuster last night as Republicans and oil-state senators, outnumbered on a critical test vote, sought to block the Carter adminstration from scoring a major victory on on its proposed oil tax.

"There's a filibuster on," proclaimed Majority Leader Robert C. Byrd (D-W. Va.) as he called on senators to cancel social engagements and stand by round-the-clock sessions until the impasse is broken.

A cartload of cots was positioned just outside the Senate chambers, drawing grimaces from senators as they milled about, waiting for a break-through in behind-the-scenes negotiations.

The stall began after Byrd and other pro-adminstration forces succeeded in blocking a move to kill a key tax-toughening proposal. Its opponents then defeated a debate-limiting cloture petition and threatened to talk for days to prevent any more senatorial nibbling at the huge profits that are expected to flow from the administration's gradual lifting of oil price controls.

After nearly a month of desultory attention to the huge tax bill, the action appeared to be getting serious. Sensing the drift, and facing a separate proposal that would tax Louisiana's oil royalties, Finance Committee Chairman Russell B. Long (D-La.) dropped a veiled hint that he might reconsider his support for the administration's Chrysler Corp. relief bill if the Senate snared Louisiana in its tax net.

Without being overly specific, Long noted that Chrysler operates two plants in Missouri, home state of the royality proposal's chief sponsor, Sen. pJohn C. Danforth (R).

The adminstration's tentative victory came on a 53-to-44 vote against tabling, or killing, a proposal to add $31 billion to the roughly $155 billion that the current Senate version of the House-passed bill is expected to produce through 1990.

The proposal would impose a so-called "minimum tax" of 20 percent on the decontrol of windfall from various categories of oil that the Senate bill would exempt from the tax, including newly discovered oil and oil that is expensive to produce.

Proponents of a tougher tax had hoped to follow the anti-tabling vote with adoption of a debate-limiting cloture petition and then fast approval of the minimum-tax amendment.

However, the cloture move failed, freeing the oil industry's friends in the Senate to block action with talk in hopes of extracting concessions from the tough-tax bloc. "It looks like. . . we're going into the filibuster season in the Christmas season," said Minority Whip Ted Stevens (R-Alaska).

Byrd was holding firm for the 20 percent minimum levy. Bob Dole (R-Kan.), a leader of the opposition, was pushing to keep newly discovered oil exempt, while raising the levy on other categories of oil to 25 percent. With much more taxation, the oil industry could end up like Chrysler, seeking a federal bailout, Dole said.

As midnight passed, Republican senators declaimed on energy matters before a corporal's guard of yawning colleagues. Others had vanished, presumably to office sofas or cots squirreled in various hideaways. Two dozen cots wee neatly arrayed just off the Senate floor in an elegantly appointed caucus room, and a generous supply of blankets was on hand.

At the center of the filibuster curtain-raising ceremonies was Long, whose opposition to the minimum tax paled by comparison to his anguish over Danforth's amendment to tax royalties from oil produced on state-owned lands. Danforth's amendment would cost Louisiana about $1 billion over the next decade, according to Danforth's figures.

Long appealed to other senators not to impose a "gag rule" (cloture) on such an important "constitutional issue" and said he needed lots of time to discuss it.

The only issues now, said Long, are "whether we should have a $155 billion tax of a $185 billion tax -- and what we are going to do about states' rights."

If there were any doubts about Long's resolve to keep Danforth at bay, they were, buried under a voluminous stack of about $300 amendments to the Danforth amendment that Long handed up in batches to the clerk's desk.

Most of the amendments would simply change one date or one word, such as substituting "the" for "a." One would insert the words "constitutionally dubious" before a reference to Danforth's royalty tax.

Another vote to choke off debate is scheduled for today, with the outcome uncertain. In yesterday's cloture vote, the 53 votes for it were seven votes short of the 60 needed to restrict the Senate to no more than 100 hours of debate and maneuvering on the whole tax bill.

On the tax increase proposal, the administration's margin was more comfortable than some senators expected. It indicted the Senate may be willing to go into a House conference with a total revenue figure high enough to assure a compromise -- possible well over $200 billion -- that would satisfy President Carter.

The House bill would produce an estimated $277 billion by 1990, only a little short of what the administration wanted. The Senate Finance Committee proposed a tax that would produce $138 billion, which the Senate has so far raised to about $155 billion.. Majority Leader Byrd, who played the key role in yesterday's administration victory, wants the Senate to approve a tax that would yield a least $185 billion. The minimum-tax amendment would assure this.

Carter and his Cabinet have personally lobbied for the minimum tax, regarding it as the most important single issue in the bill. Energy Secretary Charles Duncan and Treasury Secretary G. William Miller watched the vote from the galleries.

Virginia senators supported the move to kill the minimum tax, while Maryland senators opposed it. Similarly, Maryland senators supported cloture to limit debate, while the Virginians opposed it.