Sen. John C. Culver (D-Iowa), a leading advocate of arms control in the Senate, yesterday challenged Sen. Henry M. Jackson (D-Wash.) in an unexpectedly sharp confrontation in the developing political warfare over new strategic arms limitation talks (SALT) with the Soviet Union.

Barely nodding in the direction of traditional senatorial meeting, Culver proposed that Jackson's Armed Forces subcommittee henceforth no longer serve as the forum for consultative closed-door hearings with administration officials on the new SALT agreements now taking shape in negotiations in Geneva.

Culver also implied that Jackson's staff has been at least partly responsible for "a torrent of leaks" on the SALT talks in recent weeks - leaks Culver said could sabotage the negotiations. He called for a Senate investigation of the leaks.

Jackson's staff denied responsibility for leaks. Jackson said he "deeply regretted" an Evans and Novak column published Friday purporting to describe a secret meeting of his subcommittee - one of the leaks Culture criticized.

Yesterday's flurry of statements and counter-statements did not bear on the substance of the SALT negotiations, but rather on the struggle - "guerrilla war," as one participant described it - which has begun between supporters and critics of new agreements.

Pre-arms control senators in private have lately bemoaned their inability to mount an effective opposition to Jackson and his staff long recognized as resourceful and effective proponents of a hard-line position on arms-control issues. Culver apparently decided to try to do something about this yesterday.

He chose the issue of leaks to make his first public statement. Detailed accounts of the agreements being negotiated with the Soviets have been leaked, first to The New York Times and then to other publications, and there have also been published accounts purporting to described closed-door sessions of Jackson's subcommittee.

Last week, Paul Nitze, a former SALT negotiator who has emerged as a leading hard-line spokesman, released numerous additional details on the negotiations. The new issue of Aviation Week contains further details and analysis obviously based on classified information.

Culver called this "the hemorrhanging of top-secret details of the most important and sensitive national security negotiations of our times." He added: "Such leakage has the effect, if not the intent, of sabotaging SALT."

Senate sources sympathetic to Culver said he had found a productive issue, since Jackson's staff has long been regarded as every effective in its ability to influence news accounts. Other Senate aides do not hide their admiration for the way Jackson's office advances the senator's interests by influencing reporters' articles.

Several sources said the atmosphere at yesterday's hearing in Jackson's subcommittee - the third at which Secretary of State Cyrus R. Vance has testified on the SALT negotiations - indicated that the recent flurry of leaks has produced new sympathy for the administration and its position.

In several small ways Jackson yesterday yielded some of the ironclad control he had previously maintained over these hearings with Vance. For the first time, most of the senators present brought along aides - previously only Richard Perle. Jackson's principal aide, and several Armed Forces Committee staffers had been permitted in the hearing room.

Also for the first time yesterday, Jackson permitted State Department officials to read the transcripts of Vance's two apperances last week. These transcripts have been very closely held.

Culver's proposal that Jackson's subcommittee no longer by the forum for Vance's consultative appearances on the Hill is an indication of the view - widely held among pro-arms-control senators - that Jackson now has an excessive influence. Culver suggested that a 25-member SALT advisory committee of the Senate appointed last year by then Sen. Walter F. Mondale would provide a better forum.

Several Senate sources who oppose Jackson on arms-control issues expressed the hope that yesterday's changes in the hearings indicated that Jackson was now on the defensive.

Jackson yesterday expressed "deep regret" about the Evans and Novak column purporting to describe vance's first secret appearance before Jackson's subcommittee last month.

The column said "Vance's performance . . . bordered on the disastrous," and described alleged exchanges between Vance and several senators that the columnists said, heightened the possibility that the Senate would reject the new arms pacts.

Jackson said the column was "inaccurate and misleading" and he praised Vance's testimony.

Two weeks ago a source on Jackson's staff gave The Washington Post an account of Vance's first appearance that closely paralleled the Evans and Novak column, though with some different details. That source said Vance had been ill-prepared, that he was "out of it" on the details of SALT. and that many senators present had been upset or even angered by his testimony.