The results represented at least a partial affirmation of Merkel's emphasis on Germany's stability and economic prosperity at a time of upheaval elsewhere around the globe. They clear the way for her to extend her 12-year stewardship to 16, which would tie the record for postwar Germany.
But the groundswell of support for the far right upended any notion that Europe's largest economy and most important geopolitical player is immune to the populist currents roiling other democracies across the West. It also revealed the depths of lingering resentment toward Merkel's 2015 decision to welcome more than a million asylum seekers amid the European refugee crisis.
Taken together, the results are likely to complicate Merkel's ambitions for her final term, with thorny coalition negotiations to come in the weeks ahead, while giving the bomb-throwing Alternative for Germany party (AfD) a prominent platform to influence politics in a country where civility and consensus have long been prized.
"It's a huge change in the political landscape," said Hans Kundnani, a senior fellow at the German Marshall Fund. "It will change the nature of debate in the Bundestag."
The anti-immigration, anti-Islam AfD, founded just four years ago, was on pace to take third place with 13 percent of the vote, according to projections. That's nearly triple its 2013 result, and matches the outer limit of its support in pre-election polls.
True to its combative reputation, the party wasted no time in vowing to use its voice in the Bundestag, the larger house in Parliament, to cause trouble for the government — including by investigating Merkel's refugee-related decision-making.
"We're going to hunt Merkel," Alexander Gauland, the AfD's co-leader, told cheering supporters. "We're going to get Germany back."
Projections showed that Merkel's center-right Christian Democratic Union won approximately 33 percent — down several points from what most pre-election polls had predicted, and a full seven points lower than its result in 2013. The vote share was among the lowest ever for the party, which has governed Germany for most of its postwar history.
"We're not going to beat around the bush. We were hoping for a better result," a restrained Merkel told supporters at CDU election night headquarters in central Berlin.
While promising to remain as chancellor, Merkel also acknowledged the "strong challenge" from the AfD and its voters.
"We want to win them back," she said. "We want to address their fears and concerns."
The center-left Social Democrats (SPD) were expected to place a distant second with about 20 percent of the vote, a dismal result for Merkel's coalition partner of the past four years.
"Taking in so many refugees split the nation, that has become clear," SPD leader Martin Schulz told supporters. "We didn't manage to convince a segment of our society that our country is strong enough to handle the task."
Schulz said the party would not join another government, and would instead go into opposition to offer voters a clear alternative to the CDU for the next election. The decision leaves Merkel with few options for reaching a governing majority.
Her only viable one may be a so-called Jamaica coalition, a reference to the colors of the three parties that would take part: Merkel's CDU, the pro-business Free Democrats and the Greens.
But that could be a tricky arrangement given the two junior parties' differences on key economic and foreign policies. The Greens, in particular, said Sunday they would put strict conditions on their participation.
"We don't want to govern just to govern," said Cem Özdemir, co-leader of the Greens.
Kundnani said the Greens — who have already had a run in government with the Social Democrats — will be reluctant to do a deal that could force them to sacrifice core principles.
"This is going to be a very right-wing government one way or another," he said.
Still, there will be significant pressure on the parties to strike a deal. Without one, Germany could be forced to hold new elections, an outcome that the mainstream parties will fear would only further strengthen the AfD.
Sunday's vote continues a Europe-wide trend toward fragmentation, as the two major parties both lost support and the smaller parties gained. The Bundestag will now be crowded with six parties that span the ideological spectrum from far left to far right.
The result follows a relatively low-key campaign in which the major parties largely ducked the chance to challenge Merkel. Despite the lack of pre-election intrigue, turnout was higher than usual, with over three quarters of Germany's 61 million voters casting ballots from the Baltic Sea to the Alps.
Exit polls showed that many of the AfD's voters were casting ballots for the first time, while supporters of other parties said they showed up to try to block the party.
"We have to counter the right-wing people," said Abdessamad Mendoui, a 75-year-old who immigrated to Germany 55 years ago from Morocco, and who walked slowly on Sunday to his polling place in the western city of Frankfurt.
To Merkel's backers, the election was a chance to endorse German stability at a historically fraught time.
"It doesn't look good in the world. If you listen to that guy in America and also in the East with his atomic weapons, you get scared," said Elida Baller, 84, referring to President Trump and Kim Jong Un, the North Korean leader.
But the AfD's support was higher than many Germans had thought possible in a country where the memory of extreme-right-wing government is a source of profound national shame.
"For me, the AfD is a Nazi party," said Hannelore Weimann, 81, after voting in Berlin. "I experienced World War II and I have seen the history, and I don't understand how people can vote for a party like this."
The party was founded in 2013 as a protest against European bailouts for Greece, and that year it barely missed the 5 percent cutoff for making it into the Bundestag.
But its popularity has surged on the back of resistance to Merkel's decision in 2015 to open the country's borders to hundreds of thousands of asylum seekers fleeing war, oppression and extreme poverty.
The party was especially strong in the former East Germany, winning 23 percent there — compared with just 11 percent in the west.
Jens Töpfer, 36, an engineer who cast one of his two ballots Sunday for the AfD, said he is afraid that Germany is being overrun by refugees. And that could bring dire consequences for the next generation, he said, pointing to his 2-year-old son.
"Refugees have a different culture that doesn't fit in here," he said. "They should go back where they came from and fight for their freedom and reconstruction."
Germany's other major parties were united in speaking out against the AfD. Schulz called the far-right party "the gravediggers of democracy."
The AfD's supporters have countered that Merkel is the real threat to democracy, and that her 12-year run — now likely to extend to 16, tying a record set by Helmut Kohl — has stifled debate.
It's a view that is shared even by non-AfD voters who have grown frustrated by the robust consensus of the political establishment.
"Every four years we can vote for the person who drives the train, " said Hubert Lützelberger, 67, a former accountant who cast one of his votes for Die Linke, the far-left party. "But we can't change the direction of the train."
Isaac Stanley Becker and Alexandra Rojkov in Brandenburg an der Havel, Rick Noack in Berlin, and Souad Mekhennet in Frankfurt contributed to this report.