James Francis Reilly Sr., 72, an attorney, a former chairman of the D.C. Armory Board and a civic activist who worked to make the government more responsive to the needs of the District of Columbia, died of congestive heart failure Sunday at his home in Washington.

Mr. Reilly was a member and chairman of the Armory Board, which operates RFK Stadium, at the time the faclity was dedicated in 1961. It was, perhaps, the most visible of the services he performed for the city, but it may not have been the most difficult or the farthest reaching.

In 1962, and 1963, he was one of six citizens who formed an informal committee to persude President Kennedy that Washington needed its own man in the White House. The committee made its approach through Attorney General Robert F. Kennedy and stated its case in these terms:

"The city's Negroes, conscious of that fact that they are no longer a minority, will not be put off . . . White hostility to the former minority is developing at the same time. . . Indeed, it is more likely that any furture trouble will develope out of clashes between the two groups rather than from within the Negro community."

The committee pointed that the city government was effectively in the hands of conservative Southern congressmen, that the plight of black youngsters here was bleak and worsening and that a growing number of blacks were disenchanted with the sluggish performance of the city administration in dealing with these and other problems.

The result was that Robert Jennedy persuaded the president to appoint an aide whose special responsibility would be Washington metropolitan affairs. Charles Horsky, a well-known attorney, was the first person to hold that post.

In the early 1960s, Mr. Reilly also was a member and chairman of the Leagal Aid Agency. This was a privately funded organization that provided legal services to defendants in criminal actions who could not afford to hire lawyers. Its functions now are performed by the Public Defenders Service, which is funded by the federal government, Mr. Reilly was chairman of the Legal Aid Agency when Criminal Justice Act was passed in 1964. The law provided for public defender groups throughout the federal court system. Mr. Reilly was a supporter of the legislation.

In the late 1960s, Mr. Reilly was a member of the criminal justice committee of the U.S. Court of Appeals for the Distirct of Columbia. The committee made recommendations for what became the D.C. Court Reform and Criminal Procedure Act of 1970. This law gave to the D.C. Superior Court general jurisdiction over criminal and civil matters that previously had been handled by the U.S. District Court here.

Mr. Reilly served on numerous other official bodies, including the committee on admissions and grievances for the federal courts, the U.S. Postal Forum, which was set up to recommend ways to improve the Postal Service, and the advisory board of the old Post Office Department.

At the same time, he carried on a private law practice in which he specialized in administrative and regulatory matters. He opened his office here in 1945 and remained active in it until his retirement in 1978.

A native of Pittsburgh, Mr. Reilly grew up there and in Washington. He attended Mount Saint Mary's College in Emmitsburg, Md., and earned his law degree from the Columbus University Law School, which now is part of Catholic University.

He was in private practice breifly in the late 1930s. From 1938 to 1940, he was an assistant corporation councel in the D.C. government. From 1940 to 1944, he was an examiner of the Civil Aeronautics Board. In 1945, President Franklin D. Roosevelt appointed him to the D.C. Public Utilities Commission, now the D.C. Public Service Commissionn. The agency regulates utility rates, taxi fares and similar matters.

In his private practice, Mr. Reilly's major clients included United Airlines and the Potomac Electric Power Co.

Mr. Reilly was a Democrat and a delegate to five Democratic national conventions.

He was a member of the parish of St. Ann's Catholic Church and engaged in fund-raising activites for the Archdiocese of Washington.

His wife, the former Mildred Ricciardi, died in 1968. His first son, Anthony, who was born in 1938, died in infancy.

Mr. Reilly's survivors include for daughters, Patricia R. Browning of Fayetteville, N.C., Kathleen R. Quinn of Washington, Francesca Reilly-McDonnel of Arlington, and Anne Marie Reilly of Washington; two sons, James Francis Jr., of London, England, and Peter R., of Bethesda; one sister, Mary R. Anderson, of Syracuse, N.Y., and 16 grandchildren.

The family suggests that expressions of sympathy be in the form of contributions to Mount Saint Mary's College, Emmitsburg, Md.