A New York judge today ruled against the parents of a 25-year-old Silver Spring woman who charged that their daughter has been held against her will here by members of the Unification Church of the Rev. Sun Myung Moon.

The decision to allow Barbara Ann Larson to remain with the church marks the fourth time since 1975 that distraught families have challenged Moon's organization, and lost.

Bruce Brown, director of legal services for the church, said at a recess in New York State Supreme Court in Manhattan that such challenges are relatively rare, because it is extremely difficult to prove that an adult is being held against his or her will in the face of testimony to the contrary. Larson testified that she lived voluntarily with church members here.

Larson testified during the three-day hearing that she wanted to remain with the church, to "pursue my First Amendment right of freedom of religion. '

Miss Larson had been a member of the church for 7 years, when a St. Louis judge awarded custody of Miss Larson to her parents last fall. She was flown by professional "deprogrammer" Gifford Capellini to a "rehabilitation center" in Sweet Valley, Pa., where she spent the next seven weeks. The center has been established to help people leave groups like the Unification Church.

Her parents contended that she was kidnaped by the church from a Christmas party and flown back to New York City to live with the church. Miss Larson vehemently denied during the hearing that she had been kidnaped from Sweet Valley by church members.

At issue in the case was whether Miss Larson was mentally competent to decide if she wants to remain in the church. While her parents contended that the church had robbed her of her free will to make such a choice, she and lawyers for the church said that she is fully competent to decide that question.