An eleventh-hour compromise proposed by European neutral nations appeared today to break a longstanding East-West deadlock at the Madrid preparatory session and bolstered hopes that the 35-nation Conference on Security and Cooperation in Europe would open here on schedule Tuesday with a formal agenda.

Fears that the formal opening would prove to be an embarrassing demonstration of lack of East-West dialogue were diminished by an agenda paper from neutral Sweden that was well received by the Soviet Bloc and the Western caucus.

Since the preparatory sessions opened here Sept. 9, Western delegates consistently have been rebuffed by the Soviet Union in attempts to ensure that the conference will allow for a thorough examination of human rights violations and a full discussion on the problems raised by the Soviet invasion of Afghanistan.

In the days prior to the U.S. presidential election, the impasse was such that there were no speeches at the plenary sessions. The election result, and a firm U.S. delegation declaration that its policy here remains unaltered, appeared to spur the neutral bid to achieve a compromise and a parallel Eastern Bloc decision to move toward consensus.

Ambassador Javier Ruperez, head of the host Spanish delegation, said he believed an agenda would be ready by the opening date although he envisaged further tough bargaining. He termed the Swedish proposal "the very last chance" to get the conference off the ground with any possibilities of meaningful dialogue.

The Swedish paper, described as a full timetable for coming debates, was drafted after consultations last night with Austria, Finland and Switzerland. These countries have acted independently from those of NATO and the European Community in the preparatory sessions.

Meeting one of the main Soviet demands, the Swedish paper sought to limit human rights debate, technically known as the review of implementation of the 1975 Helsinki final agreement, to the weeks before the Christmas recess. It provides for discussion of a European disarmament conference, which is a main Eastern Bloc interest, in late January when the conference is scheduled to reconvene.

But the compromise formula went well beyond previous Soviet-backed proposals that would have limited human rights discussion to as little as 2 1/2 weeks -- or, as one U.S. source said, to a ridiculous 79 minutes per delegation.

Signaling the Eastern intent to compromise, the Romanian delegation welcomed the Swedish proposal as "a good basis of discussion."

Conference sources said a factor in pushing the conference toward consensus could have been the brinkmanship displayed yesterday by U.S. delegation cochairman Max Kampelman, who said the U.S. election would in no way change the delegation's approach. He said the U.S. delegation "has insisted and will continue to insist on adequate time for a through review of implementation" of human rights.