White House press secretary James S. Brady, under treatment for pneumonia, is improving rapidly with the help of antibiotics, according to doctors at George Washington University Hospital.

"He's really doing pretty darm well,' said Dr. Dennis O'Leary, the hospital's dean for clinical affairs."There's nothing to do but wait for this to resolve. . . .It'll take a few days to clear."

Doctors began administering two antibiotics intravenously on Wednesday, and added a third Thursday after a chest X-ray revealed pneumonia in the lower lobe of Brady's left lung. O'Leary said they were encouraged to note that Brady's temperature rose only to about 101 degrees Thursday night, several degrees lower than on the two preceding nights.

He added that Brady has no cough or trouble breathing, and that his doctors are not sure whether his pneumonia is caused by bacteria or by a virus. He said that if Brady continues to make progress they may soon eliminate one of the three antibiotics he is receiving.

Brady, who has suffered a succession of complications since he was shot in the head March 30 during an attempt on the life of President Reagan, was rested and in good spirits yesterday morning despite a slight fever. When O'Leary mentioned his temperature, he said "Yeah, I got pneumonia," jokingly pronouncing the "p." O'Leary, laughing, said Brady added another comment, "but it was not quotable."