Three men yesterday contradicted Supreme Court Justice William H. Rehnquist's account during his 1971 confirmation hearings of the role he played in a controversial Republican "ballot security" program in Phoenix in the early 1960s.

Rehnquist, President Reagan's nominee for chief justice of the United States, had said he was in overall charge of the programs, but declared that he never personally engaged in challenging the credentials or literacy of Democratic minority voters.

But two of the witnesses interviewed yesterday said that they saw Rehnquist challenging black and Hispanic voters in the Arizona city. The third, a former assistant U.S. attorney who was sent to one precinct to investigate complaints of harassment, said Rehnquist was definitely a member of a group that was "aggressively challenging minority voters by looking at them coming up in line and saying, 'You don't know how to read.' "

The former prosecutor, James J. Brosnahan, now a senior partner in a San Francisco law firm, said he had no doubt that it was Rehnquist he saw and that it was not a case of "mistaken identity" as the majority report of the Senate Judiciary Committee suggested in 1971 when the charges were first aired.

Brosnahan was emphatic in attacking the "mistaken identity" suggestion.

Rehnquist could not be reached for comment. Rehnquist's Senate confirmation hearings are scheduled to begin Tuesday.

The three witnesses, Brosnahan, a self-described liberal Democrat, and two Phoenix residents, Charles W. Pine and the Rev. Snelson W. McGriff, were cited in a special report by the Nation Institute, a liberal group affiliated with The Nation magazine. All three confirmed their remarks and provided additional details yesterday in separate interviews.

Charges of voter harassment against Rehnquist first surfaced at the 1971 hearings on his appointment to the Supreme Court.

Rehnquist denied any improprieties at the 1971 hearings, both in direct testimony and later in a lengthy letter to the Senate Judiciary Committee.

In his letter, Rehnquist said that he had been in charge of GOP "ballot security" programs and similar efforts conducted by lawyers' committees in the biennial elections in Phoenix from 1958 through 1964. However, he said, that "in none of these years did I personally engage in challenging the qualifications of any voters."

Pine, a Phoenix public relations man and state Democratic Party chairman from 1972 to 1976, said he saw Rehnquist on one occasion approach two prospective voters waiting in line outside the polling place of a predominantly black and Hispanic precinct and ask them if they were qualified voters.

"He challenged one and then another," Pine recalled yesterday. "Both were black men and both left the line." He said Rehnquist was with one or two other men and they left together after the incident, with Rehnquist driving. Pine said Rehnquist was part of what he called Republican "flying squads" going from precinct to precinct in heavily Democratic areas, either in 1962 or 1964.

Told of Rehnquist's 1971 disclaimer of any such activity, Pine said: "If he wants to say that, he can. But I disagree." Neither Brosnahan nor Pine spoke up in 1971, saying they thought about it, but decided not to bother. McGriff submitted an affidavit in 1971 saying he saw Rehnquist challenging voters at McGriff's polling place, located in the Bethune school. McGriff said he thought it was in 1964, but was not positive.

"If it wasn't him Rehnquist , it was his twin brother," McGriff said. "He had a card in his hand . . . . He'd say, 'Read this' . . . . I looked him right in the eye. I wasn't going to read that card. I was going to put my fist in his mouth."

Rehnquist told the Senate committee that he strongly disapproved of "any scattergun use of literacy challenges" and on one occasion in 1962 went to the Bethune precinct to tell one overzealous Republican challenger there to stop demanding that voters prove their literacy by reading printed passages from the Constitution as they waited to vote. Literacy challenges were later prohibited by the Civil Rights Act of 1964, but they were permissible in Arizona before then so long as they did not amount to harassment.

Brosnahan, however, said there were enough complaints about the GOP challenges at the Bethune precinct in 1962 that he went there with an FBI agent to investigate.

"Brosnahan said he found a small "group" of Republicans, including Rehnquist, there challenging voters on a random basis, asking Hispanic voters if they could read English and black voters if they could read at all. "They would do this right in line, rather than getting a person off to the side," Brosnahan said. "Telling one person after another, 'You can't read,' is an aggressive thing to do . . . . "

"My best recollection is that he Rehnquist was challenging voters," Brosnahan said, "but that was 1962 and this is 1986. I know he has denied that, but I have asked myself in fairness what I can remember."

"This was a group that deliberately set out to discourage and harass minority voters on the subject of whether they could read. The concept that Mr. Rehnquist was somehow supervising and making it better is one that conflicts with my recollection. I am very firm about that," Brosnahan said.

The 1971 Senate Judiciary Committee report dismissed the voter harassment charges as "wholly unsubstantiated," but a Democratic minority complained that the allegations had not been adequately investigated or effectively refuted.

Staff writer Saundra Saperstein contributed to this report.