Republican Fairfax County Board Chairman John F. Herrity had declined all summer long to debate Audrey Moore, his Democratic challenger in the Nov. 3 election. When he changed that strategy this week, the long-held acrimony between the contestants came immediately to the fore.

"Audrey, I don't know if you are a flip-flop artist, or you just forget," Herrity said to Moore yesterday morning at the Temple Rodef Shalom in McLean. "But you voted against the Dulles Toll Road," a major arterial that reached capacity almost the day it opened.

Shaking her head as he spoke, Moore challenged Herrity's assertion and told the crowd of 45 that she voted in favor of the Dulles Toll Road. "I have the minutes of the board meetings in my car," she said.

According to campaign advisers, Herrity wants to portray Moore as opposed to growth and roads in a county where development, traffic and a demonstrably inadequate road network have become the big issues.

The hour-long debate included questions ranging from high school birth control clinics to tax increases. On almost every issue, the two candidates gave similar answers, yet their personal styles and criticism of one another made them appear miles apart. Both have been members of the Fairfax board since 1972.

Herrity elaborated later in yesterday's debate on the toll road issue and said that Moore opposed the road by voting against a $5 million county appropriation for the project on Dec. 13, 1982.

Moore said she "absolutely" supported the road leading from the Capital Beltway near Tysons Corner to Dulles International Airport. However, she said, she opposed the appropriation "because I didn't think the county should have to pay for it."

"It was the key vote," said Herrity, who is seeking his fourth term as chairman. "If she didn't have enough guts to pay for it, she can't say she supported it . . . . She shouldn't try to cover it up. She should just admit it and go on from there."

Of the seven times the toll road came before the board, Moore voted for it six times, according to her spokeswoman, Janice Spector.

Yesterday afternoon, both Herrity and Moore were scheduled to address the Springfield chapter of the National Association of Retired Federal Employes, but at different times, a format that has been common throughout the summer, to Moore's dismay.

Moore called the association yesterday and asked if she could show up when Herrity was scheduled to arrive. Mike Michel, a vice president of the association, said he then called Herrity, who declined a joint appearance. Asked why he preferred separate appearances, Herrity said: "It's a personal problem. The whole thing is a personal problem. We all have those."

During the morning debate, Herrity and Moore both said traffic was the county's major problem, that they disapproved of high school clinics dispensing birth control devices and that they favored tougher campaign financial disclosure laws for public officials. Herrity was convicted of a misdemeanor for violating a state disclosure law last year after he failed to reveal his business partnership with a developer before voting on the developer's land-use application.

James S. Morris, a Herndon resident and businessman who owns a residential real estate firm, and R. Terry Robarge, a mortgage banker from Oakton, both independent candidates for the board chairmanship, also attended the debate.