Maryland's property tax assessment system seemed intriguing to three unlikely tax rebels, but they say that, when they challenged it in 1980, they found the intrigue was only beginning.

Throughout administrative appeals, tax authorities have refused to say how these homeowners' properties were evaluated or what the criteria were, they said. And when the residents sued in tax court, the state, rather than disclose records of tax assessment procedures, filed instead for a protective order to shield the documents. Their entire case challenging the constitutionality of the state's four-year-old, triennial assessment system has been awaiting action for the last two years.

"What we've been faced with so far at every level of government during Watergate days would be referred to as 'stone-walling,' " said Hays Gorey, one of the plaintiffs, who ironically is an attorney for the Justice Department. "Not a single document has been released. The arrogance of the state's attitude throughout this has been apalling."

Gorey's remarks came along with those of two other Montgomery County homeowners who have joined suits challenging the three-year assessment plan--Robert Cohen, a solar energy consultant, and Frank Ecker, former mayor of Rockville and most recently, former public advocate for the county's supervisor of assessments. That very office was the starting point of the trio's assessment challenges.

They all spoke at a recent meeting of a Bethesda residents group, the Whittier Woods Civic Association, an occasion in which no kind words were to be found for Maryland tax authorities. Residents joined in, complaining of a range of suspected tax atrocities, from official "fudging" to "statistical cheating" in the state's controversial triennial home assessments. Someone voiced the suspicion that "scoundrels are involved."

The three men recounted their story, as they have to a number of Montgoemry civic groups, telling of how they found themselves among the first wave of residential property owners given reassessments under the Beck Act of 1979. That law resulted in a system of dividing properties into three groups, reassessing each once every three years and spreading the change in value over the following three years. The cycle then repeats.

They appealed their tax bills, and the first step took them to Robert Rudnick, supervisor of assessments for the county. Gorey said he was asking officials for an explanation of his bill. "Their answer was a computer printout of what they had already sent" in his bill, Gorey said.

Cohen and Gorey, of Chevy Chase, met Rockville resident Frank Ecker when they all took their case to the county Property Tax Assessment Appeals Board. The three-member panel issued an opinion in June 1981 stating that it lacked the authority to adjust their bills on the grounds they had asked--which was that the triennial system created inequities.

But the decision did direct Gene L. Burner, the state tax assessor, to reexamine all property assessments, sort out any systematic unfairness, and send the board a detailed report. Instead of sending such a report, Burner sent the board a letter stating that he had reviewed the system and found that it was equitable, Gorey said.

Burner could not be reached to talk about that report.

In August that year, Cohen, Eckert and Gorey filed suit in county Tax Court, essentially an administrative body, alleging that the Beck Act violates the state constitution's provision that all citizens are to be taxed fairly. They also have combined the argument of another challenger, Suzanne D. Racoosin, whose case contends that all property within a class, i.e., residential, industrial, commercial, should be taxed equitably. Because the residential property owners are divided into separate groups and given differing assessments, that is a violation, the case alleges.

As part of their suit, Cohen, Eckert and Gorey have asked the state to release documents showing the basis for dividing properties into groups, and those showing what criteria were used in doing it.

State attorneys for the state tax assessor's office have filed for a protective order that would allow officials to withhold that information from those suing. A hearing on that order is set for June 29.

A county official said in an interview that the categories were set up according to the county's schedule for actual property assessment inspections. Assessors previously set values by inspecting properties one year and evaluating them statistically the following two years, the official said.

The schedule is arranged basically by election districts, the official said. The first group reassessed under the triennial system included the 2nd, 3rd, 6th, 11th and 12th districts, plus part of the 4th, 7th, and 13th. The second group consisted of the 10th district and parts of the 7th, 8th and 13th. The last group to be reassessed included the 1st, 5th, 9th and parts of the 4th and 8th election districts.

Even without the documents, the challengers say they can show the system is inequitable.

"I know simply by applying the mathematics that it's unfair," said Frank Ecker, who, for the edification of the homeowners at Whittier Woods, showed that the owner of a hypothetical $100,000 house in the first group, reassessed in 1980, would pay $600 more than the owner of a $100,000 house in the third group reassessed last year.