About 20,000 District of Columbia voters, whose names could not be found on computerized registration lists, were forced to cast special challenged ballots in yesterday's primary election, resulting in long lines and frayed tempers.

The size of the foul-up, affecting an estimated 17 percent of those trying to vote, raised the possibility that the outcome of any close races could be in doubt for days, until the challenged ballots are counted.

Angry voters, including both longtime and new residents of Washington and even several candidates, top city officials and precinct workers, found their names inexplicably missing from registration lists or listed incorrectly.

Many voters who had brought their registration cards with them were told they were not registered and would have to cast challenged ballots, for which they were required to sign an affidavit and fill out additional forms.

Some would-be voters left in frustration, according to precinct workers, while others complained that the secrecy of their ballots was jeopardized because they were required to sign special ballot envelopes containing their ballots.

In the final vote released early this morning, about 20,000 ballots were challenged out of more than 121,000 cast.

Albert J. Beveridge, chairman of the Board of Elections and Ethics, said the number of challenged ballots was higher than the board expected. He estimated 60 percent of the problems were due to "board error" in inadvertently dropping names from voter lists.

The rest of the problems, he said, were caused primarily by voters showing up at the wrong precinct either because they had recently moved without informing the board or because their district boundaries had been changed..

Despite the board's months-long effort to avoid the problems that plagued some past D.C. elections, it appeared that challenged ballots would be about triple the number in the 1978 election, when the board had roughly 5,000 challenged ballots and took 15 days to deliver a final count.

Elections board officials said they would be able by Monday to count challenged ballots.

Both Beveridge and Teddy Filosofos, the board's new executive director, said they did not think the problems were serious. "I can see we have made some mistakes. But I don't think there is that big of a problem," said Filosofos at the board's District Building headquarters, where election workers were fielding thousands of calls from disgruntled voters.

Just last week, the elections board said they had developed the most comprehensive voter list the city ever had, containing 328,000 names. The board estimated the city had only about 300,000 registered voters and said the extra 28,000 included some who had died, moved, or accidently were listed more than once.

The City Council had urged the board to develop the largest list possible to avoid voters being turned away on election day. Filosofos said the board had done so by amassing every available list of persons who had registered in recent years.

Filosofos, who took over board operations last April, said he believed much of the problem yesterday stemmed from the recent conversion of voter rolls from the city computer system, which he characterized as "garbage," to a new computer system.

Among the voters whose names were dropped from registration lists were City Council member and mayoral candidate John Ray, City Council Chairman Arrington Dixon, City Comptroller Alphonse Hill, Corporation Counsel Judith W. Rogers and other public officials, who cast challenged ballots. Ray called the foul-ups "just another example of how sorry this city government is."

Dixon said that the foul-up was "an affront to the community," and said the elections board "is more concerned with computers and systems and mechanics than they are with voters . . . . I would rather have paper ballots and mark an X instead of a computer system that doesn't work."

During the 1978 elections, candidate Marion Barry had blamed election day snafus on then-mayor Walter E. Washington. But early yesterday, before Barry's lopsided victory was evident, supporters were concerned that this year's foul-ups could cost Barry votes.

"I think this could hurt the mayor. It just doesn't make sense that they can't get this straight," said Harold P. Sanders, a Barry supporter and an architect who does business with city agencies.

"The voters are angry. I feel ashamed of the way it's gone," said Marian Tignor, the precinct captain supervising the voting at Ward 2's Syphax School in Southwest. By midday, Tignor said, 152 voters out of 1,100 were forced to cast challenged ballots, most often because their names were missing from the rolls.

"Some people got in a huff and left. One voter called the Board of Elections and said we were turning voters away. That's not true that we turned them away. We have lost some voters, unfortunately though" because they were discouraged by having to wait and fill out challenged ballots, she said.

"I'm the precinct captain and they don't have my name, said Margaret Coates, at the Slowe Elementary School in Ward 5. "People are coming up here getting angry at me, but it's not me. This is a real mess. It's those people downtown."

Tempers flared at many polling places, including the Marie Reed Learning Center, Precinct 24 in Ward 1, where some voters waited in line for more than a half hour to cast challenged ballots and harsh words were exchanged between angry voters and frazzled elections workers.

In cases where would-be voters' names failed to appear on voter lists at local precincts, or appeared with incorrect party affiliations or other problems, voters had to sign affidavits certifying that they were, in fact, registered. Then they had to put their name, address and a second signature on a special ballot envelope.

Voters then cast their ballots on computer punch-cards, which were placed inside the special envelopes. The elections board will now have to separate the special ballots and double check their master list of registered voters before deciding whether a ballot is valid.

The board said it would send letters to all voters involved in challenges, and would also publish a list of disqualified voters in the Sept. 20 edition of The Washington Post so that voters could appeal the disqualification.

After last November's election, at which many thousands of voters were turned away because their names were not on voting rolls, the City Council commissioned a study by former city auditor Matthew Watson. In a report released earlier this year, he concluded that as many as 50,000 persons might have trouble voting in yesterday's election because of incomplete, missing or inaccurate records.

Barry, who has disclaimed responsibility for elections board problems but who controls appointments to the board, allocated $100,000 for specialists to come in to aid the system. The Government Employees Insurance Corp. volunteered its services and reconstructed the voter rolls last May, but GEICO officials said the board still had more work to do.

Ernest Huguet of Ward 2 walked out of the polls without voting. He had waited to cast a challenged ballot and was told he would have to sign his special ballot envelope. Huguet said he wouldn't vote because his ballot was therefore not secret