Retired Army colonel Ernest Calhoun Norman, 84, a former deputy military governor of Bavaria and post commander at Fort McNair in Washington, died Thursday at the Veterans Hospital in Washington after a stroke. He lived in McLean.

Col. Norman, who was born in Atlanta, began his 36-year career with the Army Field Artillery after graduating from the U.S. Military Academy in 1918. He saw combat in France during the last days of World War I and spent the years between the wars at various Army bases in this country and abroad.

In 1932, he received a degree in history from the University of Chicago.

After the Allied invasion of France in World War II, he commanded the 408th Field Artillery Group in Europe. At the war's end, he became deputy military governor of Bavaria and later was commanding officer of a headquarters area command in eastern Germany.

Col. Norman returned to this country in 1947 as post commander of Fort McNair. He later served in the Pentagon as chief of the artillery branch of the Army's career management division.

He retired from the Army in 1954 and worked as a stockbroker in Washington before retiring a second time in 1958.

His decorations include the Bronze Star and three Battle Stars.

His first wife, Sylvia Norman, died in 1963.

Survivors include his wife, Curtis Madison Norman, of McLean, whom he married in 1967; a brother, Clarence E., of Raleigh, N.C.; three sisters, Eleanor Walker and Nina Norman, of Orlando, Fla., and Dorothy Oldham, of Sarasota, Fla; three stepdaughters, Barbara K. Barnes and Kathleen K. Sturino, of Laurel, Del., and Edna K. McPherson, of Tucson, Ariz.; 17 step-grandchildren and three step-great-grandchildren.

The family suggests that expressions of sympathy be in the form of contributions to the Army Distaff Hall in Washington.