Malka Tryna Waugh, 47, who taught mathematics at John Fitzgerald Kennedy High School in Silver Spring for the past 10 years, died of respiratory failure Saturday at Sibley Memorial Hospital. She lived in Silver Spring.

Mrs. Waugh had taught mathematics in Montgomery County public schools since 1966. Before moving to the Washington area in 1962, she was a volunteer teacher in Dallas, Tex.

A native of Chicago, she graduated from the University of Illinois and did postgraduate work at the University of Maryland.

Earlier this year, she sponsored the JFK High School junior class team, which won its first round of competition on the local television quiz program, "It's Academic," last September.

Mrs. Waugh was membership chairman of the Montgomery County Math Teachers Association and building representative of the Montgomery County Federation of Teachers. She was a member of the American Federation of Teachers, the National Council of Teachers of Mathematics, the League of Women Voters and the Zionist Organization of America.

Survivors include her husband, David A. Waugh, and a son, Steven J., both of Silver Spring; a daughter, Rachel Anne, of Los Angeles; her mother, Esther Mirvis of North Miami Beach, Fla., and a brother, Leonard Mirvis, of Laguna Hills, Calif.

The family suggests that expressions of sympathy be in the form of contributions to the Hillel Foundation at the University of Maryland in College Park.