The nomination of Adm. Elmo R. Zumwalt Jr. to an arms control advisory board has been stalled and perhaps killed by Sen. John W. Warner (R-Va.), who has raised questions about giving Zumwalt, now a syndicated newspaper columnist, access to sensitive national security documents.

Senate aides said yesterday the objections raised by Warner and his collegue, Sen. Harry F. Byrd Jr. (Ind-Va.), make it "very difficult" for the Reagan administration to put Zumwalt, a Democrat who backed Reagan in 1980, on the 14-member general advisory committee of the Arms Control and Disarmament Agency.

But one of the aides said that the White House is still pushing the nomination, which it had agreed to resubmit to the Senate last week after the two Virginia senators asked for a second vote on the appointment. This week Warner asked for further delay.

The Senate had confirmed the nomination of Zumwalt and 13 other appointees to the advisory board last week, catching both Byrd and Warner unaware. Zumwalt, a McLean resident, ran as a Democrat against Byrd in 1976 and criticized Warner's performance as secretary of the Navy in his memoirs and during Warner's campaign for the Senate in 1978.

In a letter to Senate Foreign Relations Chairman Charles Percy (R-Ill.), Warner said he had concerns about the potential conflict of interest posed by Zumwalt's presence on the committee. Since 1978, Zumwalt has coauthored a weekly column on defense issues with retired Adm. Worth H. Bagley.