Rudy P. Hertzog, 81, who retired as associate chief counsel of the Internal Revenue Service in 1968 after 46 years with the agency, died Tuesday at the Bethesda nursing home in Chevy Chase following a stroke.

Mr. Hertzog, who was born in Carrolltown, Pa., moved to Washington in 1921 and began his career with the old Bureau of Internal Revenue a year later. (The Bureau of Internal Revenue was reorganized at the Internal Revenue Service in the late 1950s). While working as an auditor, he attended the law school of Columbus University, which now is part of Catholic University, and earned his law degree in 1927.

Over the next 41 years, the positions he held included that of director of the Tax Court Division. From 1953 until his retirement, Mr. Hertzog was responible for all IRS litigation in U. S. Tax Courts around the country. He was named assistant chief counsel in the 1950s and in the 1960 was appointed associate chief counsel. On a number of occasions he was acting chief counsel.

Earlier in his career, Mr. Hertzog was the division counsel in Kansas City for two years and in New York City for five years. But most of his work in the Office of the Chief Counsel of the IRS in Washington and he worked in IRS headquarters here continuously from 1953 until his retirement.

After retiring from the government, Mr. Hertzog went into the private practice of tax law. He was counsel to the Washington law firm of Bird & Tansill for many years. When that firm merged with Ober, Grimes & Shriver of Baltimore in 1979, Mr. Hertzog was counsel to that firm. He held that position until his death even though he was unable to practice after 1979 for reasons of health.

Mr. Hertzog's first wife, Helen McGraw Hertzog, died in 1974. A son, Donald, died last year.

Survivors include his wife, the former Dorothy Yates Kimsey of Chevy Chase; two daughters, Rita H. Schultze of Washington, and Helen H. Linhard of Towson, Md.; four stepchildren, James V. Kimsey of McLean, Eleanor O'Sullivan of Rockville, Virginia Lightsey of Boulder, Colo., and Thomas Kimsey of Crozet, Va.; 24 grandchildren, and four great-grandchildren.