Albert J. Robertson, 84, who was a former assistant postmaster general and chairman of the Federal Home Loan Bank Board, died of a heart ailment Friday at the Wisconsin Avenue Nursing home in Washington.

Mr. Robertson moved here in 1953 when he was named assistant postmaster general for finance by President Eisehower. He served in that job until 1956 when he was named chairman of the Federal Home Loan Bank Board.

At the Post Office Department he had overall supervision of its financial operations, including the accouting and budget control systems and financial reporting.

While there he also became an enthusiast of stamp collecting, and spoke often of the subject.

"Stamps have been called ambassadors, carrying with them elements of the history, customs, or beliefs of the issuing country. More than that they serve well as a medium of education for both young and old," he said in 1955 in Richmond at a ceremony marking the first sale of the John Marshall 40-cent stamp.

His reputation as a spokesman on stamps brought him some difficulty when he defended a stamp honoring Robert E. Lee. This stamp was attacked on the grounds that Lee was a traitor to his country. Mr. Robertson, a native a Minnesota, simply responded that Lee was a "great American" and that there would be those who would criticize "even the 12 Apostles" appearing on stamps.

When Mr. Robertson was appointed to head the Federal Home Loan Bank Board in 1956, it was noted that he was the first chairman in the bank's history whose background was largely in the field of commercial banking. He retired in 1961.

Mr. Robertson was born in Minneapolis and earned a bachelor's degree at the University of Minnesota.

He served as an infantry lieutenant in France duing World War I. He worked as a bank officer for 38 years in Minnesota and lowa before entering the government here.

Mr. Robertson was a member of the Cosmos Club.

He is survived by his wife, Persis W., of the home in Washington; two daughters, Persis Gow, of Concord, N.H., and Madeleine Collins, of New York City; six grandchildren, and two great-grandchildren.