Rep. Harold Washington (D-Ill.) swept to a remarkable upset victory in the three-way race for the Chicago Democratic mayoral nomination Tuesday with a grass-roots campaign that triggered a massive black turnout amid a badly split white vote.

The victory all but assured that Washington, 60, will become Chicago's first black mayor and that Chicago will become the largest city with a black chief executive. He won 36.3 percent of the citywide vote in an election that drew 1.2 million Chicagoans to the polls, the largest turnout for a local contest in 25 years.

Incumbent Mayor Jane M. Byrne, who conceded defeat this morning after stubbornly but characteristically waiting through the night for better news, was second with 33.5 percent. Cook County State's Attorney Richard M. Daley, son of the late mayor, was third, with 29.8 percent.

Washington's victory margin was 32,000 votes, large enough to rule out a recount, and the ballots have been impounded by federal marshals to prevent vote fraud.

Washington is expected to defeat Republican Bernard E. Epton, a former Illinois state legislator who was unchallenged for his party's nomination, handily in the April 12 general election. As chief executive of the nation's second-largest metropolis, Washington could help write a new chapter in the emergence of blacks as a force in northern politics.

He already has rewritten the book here on how to win an election. Outspent 10 to 1, his shoestring campaign was boosted to success by his impressive showing in four mayoral debates that were heavily covered by the media, a concerted drive that resulted in a record black registration and turnout and a grass-roots appeal to blacks, Hispanics and whites disenchanted with Democratic machine politics.

In addition, Washington benefited to an unknown but apparently substantial degree from a voter backlash against last-minute Byrne campaign tactics.

Former Republican governor Richard B. Ogilvie, who backed Byrne, warned in letters to Northside white residents that a vote for Daley was a vote for Washington. The powerful Cook County Democratic chairman, Edward (Fast Eddie) Vrdolyak, a Byrne ally, told white precinct captains last week that the contest had become "a race thing" and they must "save your city."

Both the Ogilvie letter and Vrdolyak's remarks leaked to the Chicago press, adding fuel to what Washington in his victory speech early this morning called "a fire" that ignited to forge "a working people's coalition of blacks, whites, and Hispanics who could force them to yield this city."

Byrne, whose roller-coaster political career has careened from obscurity to the pinnacle of City Hall and down to defeat in just four years, offered her support to Washington in her concession speech. She declared at a City Hall news conference that "the results are very clear. The people spoke. There should be, for the good of the city, a smooth transition."

Daley earlier pledged to back Washington. The support of Byrne and Daley gives Washington the backing of what remains of the nation's most famous big-city Democratic machine. Between them the two white candidates got about 730,000 votes to about 419,000 for Washington. He got about 82 percent of the total black vote, with about 69 percent of all registered blacks going to the polls, another record.

Washington began his political career at the age of 13 working for the Democratic organization. A former state legislator, he was elected to his second term as representative of the 1st Congressional District, which is on Chicago's Southside and fronts on Lake Michigan.

It is a stable, middle-class district, the longest-lived urban black community in the country, and since 1970 has included the University of Chicago. It has elected a black congressman since 1928, all of whom have worked within the Democratic organization to provide their constituents with patronage jobs and government services.

Washington has been the most independent, however. After being sworn into the House two years ago his antipathy to the machine by then was so great that he refused to resign his state senate seat in an effort to keep Byrne from appointing one of her machine allies as his replacement. He eventually had to yield, however.

Vrdolyak joined the pledge of support to Washington as well. Longtime Chicago political observers say that the closing of ranks has less to do with forgive-and-forget than with the machine's desire to hang onto what it has in the face of any outside threat.

"The machine is wounded after yesterday," commented political historian Milton Racove, "but the machine transcends things. They had to live with Byrne; they'll live with Harold Washington. They'd lived with Hitler if they had to."

Thirteen of the city's 50 aldermanic wards are black. Washington swept these, plus six other wards, five on the Westside. The margins were so lopsided that he was able to hold off the pluralities Byrne rolled up in the 22 wards she carried in the North and Northwest.

Many of the Southside wards Washington carried went for Byrne in her 1979 upset of Mayor Michael A. Bilandic, Daley's successor. But black support waned when she failed to keep promises to appoint more blacks to key jobs, and she further upset them with such steps as dismissing all the black members of the Chicago Housing Authority and replacing them with whites. A comparison of some of these wards shows the remarkable turnaround for Washington.

In 1979, Byrne garnered a total of 55,493 votes in five Southside wards, part of the black protest of Bilandic's failure to keep public transit rolling after an election week blizzard cut blacks off from their downtown jobs. Yesterday, Byrne got only 14,711 votes in these wards, compared with Washington's unofficial total of 94,744.

Political strategist Don Rose, who ran Byrne's 1979 campaign said today that Washington's effort "was transformed into a movement with its own momentum, out of control of normal campaign effects. It was a brush fire . . . . He could have had a better campaign and not gotten that kind of effect, or a victory."

Washington had barely $1 million to spend, which he concentrated on buttons, posters and a late television campaign to counter Byrne's $10 million campaign.

Washington entered the race only last fall, months after the other two, and his staff was disorganized. But veteran civil rights organizer Al Raby, who has been active in the Chicago movement for two decades, was brought in last month from Philadelphia.

Rose, who sat out this year's race, said the endorsement of Washington by Sen. Alan Cranston (D-Calif.), a Democratic presidential aspirant, "gave Washington credibility in the black community. They don't know Alan Cranston from Adam, but he's a U.S. senator, a white, a presidential candidate and a serious man."