Yesterday at the Army and Navy Club, that bastion of military grandeur overlooking Farragut Square, news of Secretary of State Alexander Haig's skirmishes with the Reagan administration raced through the chandeliered halls.

The feeling of the retired military men who have watched news for decades from inside the confines of the club's walls was: Their man Haig was right. But it was bad form for his to differ publicly with Reagan and, as a military man, he should have known better.

"If you present your case and the boss rules, there's a military rule of long standing that you don't quarrel with a decision once it's been made," said Leon Smith, a retired Navy captain, as he ate his lunch in the club's Men's Bar. "And you certainly don't quarrel with it in public. Which isn't to say that I don't think Haig may be right. He evidently feels very strongly that crisis management should come under the aegis of the secretary of state, and he's a forceful man."

Smithh joked that he typified the rigid concept of a retired military man, the kind who regularly uses the Army and Navy Club. "I eat here twice a week [in the bar on the club's first floor]. I always sit in the same place [at the table for two behind the first pillar, with his back to the bar], order a bowl of soup and the same sandwich [tuna on white with one pickle, two tomato slices] and the same drink [draft beer]."

Smith's sentiments about the Haig affair could be heard elsewhere in the bar, in the grill, in the dining room, library and lounge of the musty club where legends of war have been told since the brick Renaissance-style building was built.

"The boss is the boss, so if he wants to delegate authority in a certain way, I assume it's his privilege; but hopefully they'll paper it over soon," Walter Myers, a retired lieutenant colonel, said from the corner of the first floor lounge, where he was perched in a high-back chair looking over business papers.

"All the military people think really highly of Haig. Everybody here supposes Haig's pretty strong. It shouldn't have been made public, but give it time - this administration is new yet," said retired Gen. Erle Cocke after he filled a plate from the grill's newly installed salad bar, aimed at keeping the formerly athletic members trim.

Cocke, who quickly tells his former rank in the service but declines to disclose anything else about his background, is like many of the retired military men who use the downtown facility. Now based in Georgia, where he is head of an insurance comapny, Cocke spends almost half his time in Washington and always stays at the Army and Navy Club. He is the decorated military man that he appears to be: a Medal of Honor winner and holder of the Purple Heart, the Silver Star and the Bronze Star. He was three separate times a prisoner of war and three separate times wounded. Once the Germans even announced that he had been executed. He is the former national commander of the American Legion.

"Haig's my man," said Homer Brett Jr., a retired naval commander, as he straightened his bow tie and presented his card, which listed both his former military rank and his address as a box at the club.

"Haig hasn't gone overboard in public, because when he testified to Congress no decision had been made yet about who would control the crisis team," said Bob Ellsworth, a former Navy lieutenant commander, who was eating salad alone in the grill. "Haig was quite right the way he handled it -- he was still engaged in preliminary skirmishes, but then the pygmies of the White House staff took over and used the hearing to convince Reagan he had to take a stand."

Ellsworth, who is now in private business and a Reagan supporter, said this skirmish with Haig "merely continues to weaken this country's position. I haven't been overseas since yesterday [Tuesday] afternoon, but let me tell you they all think Haig is great, and this incident is just another that will make the U.S. look not very impressive in the world."