In a case that may have wide implications for women's sports, a U.S. District Court judge ruled today that a male Newport high school student cannot be barred from playing on the school's only volleyball team - an all-girls team.

A spokesman for the Rhode Island Interscholastic League, which barred 19-year-old Donald M. Gomes from the team in March, said the decision could undo affirmative-action efforts and lead to male domination of girl's interscholastic sports.

Dorothy V. Harris, director of the Pennsylvania State University Center for Women in Sports and an official of the 1980 Olympic Games, told the court during a hearing last month that allowing boys to play on girls' teams "would virtually eliminate the opportunity of girls playing competitive sports."

Lawyers for both sides said they believe this is the first federal court case in which a male student sought a place on a girls' team, although there have been frequent cases in which girls won places on boys teams.

While the decision by Raymond J. Pettine, chief judge for the District of Rhode Island, is binding only in this state, the lawyers said it opens the door for similar suits around the country.

A spokesman for the league said an appeal is likely.

At issue in the case was a conflict between law prohibiting sex discrimination and law requiring affirmative action to remedy past failures.

A. Lauriston Parks, Gomes' attorney, argued that Title 9 of the Education Amendments of 1972 bars sex discrimination in federally funded sports and that Gomes was a victim of discrimination because the only volleyball team at Rogers High School in Newport was for girls.

Robert Mann, lawyer for the league, argued that it was merely providing affirmative-action programs to end long-standing discrimination against girls in interscholastic sports.

There are 5.531 Rhode Island girls participating in interscholastic sports compared to only 204 eight years ago.

Pettine said he agreed with the league that "open competition would in all probability the majority of females to second-class positions as bench warmers or spectators."

However, he wrote in his decision, "To completely deny one sex any opportunity to participate in a federally funded activity from which either sex could benefit is an unprecedented doctrine."

He indicated that separate but equal might be a solution to the problem, but said it is up to the schools and not to the court to establish separate boys teams in such sports as volleyball and field hockey.

Gomes, a lanky senior, has been attending the volleyball games as a spectator so far this season but could get his first chance to play this week.