James R. O'Brien, 55, an assistant director of the AFL-CIO's department of social security since 1967, died of cancer Tuesday at the National Institutes of Health.

Mr. O'Brien helped develop labor policy on workmen's compensation, unemployment insurance and disability insurance.

During his years at the AFL-CIO he served as a labor representative on the National Commission on State Workmen's Compensation Laws. From 1971 to 1977, he was the labor representative on the Federal Advisory Council on Unemployment Insurance.

President Carter recently had appointed him to the National Commission on Unemployment Compensation.

Mr. O'Brien was a native of Massachusetts, and earned bachelor's and master's degrees in economics at the University of Houston.

He worked for a time for the wood-workers' union in Oregon and Tennessee, before moving here in the mid-1960s to join the staff of the American Federation of State, County and Municipal Employees.

Mr. O'Brien was a member of The Newspaper Guild and St. Catherine Laboure Catholic Church in Wheaton.

He is survived by his wife, Lillian (Patsy) O'Brien of the home in Wheaton; a stepdaughter, Mary Louise Pry, of Iowa; his mother, Sarah O'Brien, of brother, William, and a sister, Anne Carroll, all of Massachusetts, and two grandchildren.