President Reagan visited White House press secretary James S. Brady yesterday at George Washington University Hospital, where Brady is recovering from a bullet wound to the brain suffered during the March 30 attempt on the president's life.

"I'm glad I was able to come," Reagan told Brady.

"Doesn't everyone get a visit by the president?" Brady joked back from his hospital bed.

Reagan told reporters as he left the hospital that Brady was "just fine, coming along great." The president added that Brady appeared in good spirits despite his long hospitalization, multiple operations and a difficult course of therapy.

Reagan and Brady kept the tone of their 16-minute meeting light, but it was an emotional experience for the 40-year-old press secretary, who has undergone three surgeries in the course of what doctors describe as a remarkable recovery from his severe brain injury.

"He choked up a couple of times," deputy White House press secretary Larry Speakes told reporters. Brady's wife, Sarah, who was present when Reagan and White House chief of staff James A. Baker III entered the hospital room, said later that her husband was pleased with the way the visit went.

Reagan had telephoned Brady twice since being discharged from the same hospital April 11, but this was their first visit. Speakes described the meeting on the basis of accounts from Reagan, Baker and Sarah Brady.

The president brought his press secretary a jellybean jigsaw puzzle and a jar of the candies that are Reagan's favorites.

In the puzzle box were some sheets of paper with the manufacturer's name. Sarah Brady asked what were for and her husband answered: "That's who you call if you can't finish it."

The president is holding Brady's job open and told him: "We're waiting for you to get back. We need you."

"I've been watching you on the television and reading the newspapers and you're doing pretty well on your own," Brady replied.

"You rest and take care of yourself," the president said as he prepared to leave. "That's the first time you've said that to me," Brady joked.

Lined up to greet Reagan when he entered the hospital were members of the trauma team and emergency room staff who cared for him, Brady and Secret Service agent Timothy McCarthy when the three were brought to the hospital in the hectic aftermath of the gunfire outside the Washington Hilton. A fourth victim, police officer Thomas Delahanty, was treated at another hospital. Brady is the only one still hospitalized.

Philip Birnbaum, academic dean of the hospital, said Brady's pneumonia is continuing to improve: "He had a peaceful night. He remains free of fever."

On his way out of the hospital, Reagan signed the cast on the right leg of patient Chris Horne of Mobile, Ala.