The surprise outcome offered Merkel few viable options for forming a stable government, with an unwieldy and ideology-bending alliance of conservatives, libertarians and environmentally minded progressives considered the only likely course.
The results even frayed ties within her bloc, with her party's Bavarian sibling threatening Monday to set out on its own.
The implications for Europe could be vast. Long reliant on Merkel to patch it through tough times, the continent may find that her steady hand is not what it used to be.
"Foreign policy power depends on your ability to govern back home and deliver. Merkel has always been able to deliver on the European stage. That will be a lot more complicated now," said Jan Techau, director of the Holbrooke Forum at the American Academy in Berlin. "She will not be the same singular leader of the German political scene that she was before. Her position as this towering figure will be diminished."
The timing of Merkel's stumble is especially troublesome for those who had been looking to the period after the German elections as an ideal moment to more tightly integrate Europe and shore up its rickety and crisis-prone foundations.
Foremost among them was French President Emmanuel Macron, who has laid out ambitious plans for a more centralized and efficient euro zone.
Macron had viewed the coming months as a rare window for change, with the leaders of Europe's two most consequential players — Germany and France — both enjoying a fresh mandate from their voters.
Merkel, though more tentative, had reciprocated, endorsing some of Macron's ideas and expressing a willingness to work with him.
But the election results complicate that picture considerably.
"The euro needs a revolutionary step forward to be resolved," Techau said. "That was always unlikely. But it's made even more difficult now."
The problem arises from Merkel's need to watch her right flank, where she lost ground Sunday to the insurgent Alternative for Germany party (AfD) and where there's long been deep skepticism of any deal that could put German taxpayers on the hook for deficits in Europe's less prosperous south.
Her vulnerability was underscored Monday when the more conservative Bavarian sister party to her Christian Democratic Union (CDU) said it would have to reexamine the two parties' decades-old alliance.
Meanwhile, one of her would-be coalition partners, Free Democrat leader Christian Lindner, reiterated that his pro-business party would vehemently oppose any plans for a common euro-zone budget — a key element of Macron's plans.
And even within her party, where Merkel has long had near-absolute control, there were questions raised Monday that would have been unthinkable before Sunday's results.
"The Twilight of the Gods has started for Ms. Merkel," said Rupert Scholz, a former defense minister and veteran party lawmaker. "You will feel it in a few months. There will be an intensive debate within the CDU."
All of that will limit Merkel's room to maneuver and make any major European initiatives less likely, said Josef Janning, who leads the Berlin office of the European Council on Foreign Relations.
"To compromise with other states, you need a strong chancellor," Janning said. "Her leeway is reduced."
With Macron due to give a major speech on his proposals in Paris on Tuesday, French analysts acknowledged that prospects for his blueprint had dimmed, even if they had not been extinguished altogether.
"The situation we're seeing now, in terms of the next [German] coalition, is maybe the one that is the most delicate for the French," said Pierre Vimont, a former French ambassador to the European Union.
Macron has gambled much on his ability to work with Merkel to reach a deal.
On Friday, he dared go where few French presidents have ventured, by signing into law — despite significant protests — five major provisions that will deregulate France's labor market, mostly by making it easier for employers to hire and fire workers.
Officials in Berlin had signaled that such French domestic measures were a prerequisite for any broader euro-zone agreement that would loosen German purse strings.
Merkel's strength in the E.U. is such that her voice is decisive on issues ranging from Greece's fate in the euro zone to sanctions against Russia. She has outlasted her rivals and pioneered a deliberate approach to leadership that has made her assent crucial to decisions that touch the half-billion citizens of the sprawling 28-nation bloc.
But now German domestic turmoil may blunt her ambition to further integrate the continent, while weighing down Europe's agility during any future crisis.
"She's still by far the strongest European leader, but she's clearly in a phase of decline," said Stefan Lehne, a former senior Austrian diplomat who is now a visiting scholar at Carnegie Europe, a Brussels think tank. "That will weaken her authority on the European level."
Still, there remains a wide range of E.U. issues that Merkel is likely to continue to advocate, Lehne said. The election results could only reinforce her hard-edge approach to slowing migration flows while demanding that other E.U. countries take in more asylum seekers.
If Merkel's authority in Europe is starting to fade, that could leave the bloc without direction — or it may force other leaders to step up and articulate their goals.
"People have gotten so used to relying on Germany for leadership that it's actually become a bit unhealthy," said Heather Grabbe, the director of the Open Society European Policy Institute, a think tank. "Everyone complains about Germany, but they hide behind Merkel and avoid having to take positions."
Merkel's enduring strength was probably the reason for the largely muted reaction from far-right leaders in Europe.
In Hungary, nationalist Prime Minister Viktor Orban has tangled with Merkel on migration and on rule-of-law issues. But he refrained from taunting the chancellor Monday.
"This is a clear signal that European citizens have voted in favor of a policy that guarantees the conservative fundamentals of security and economic stability," Orban wrote to Merkel, an Orban spokesman said.
In Berlin, meanwhile, Merkel was hardly the only politician under pressure. The AfD, which had been celebrating its third-place finish Sunday night, was sobered Monday morning by the surprise announcement by one of the party's most visible figures that she would not join the party's bloc in the Bundestag, the lower house of the German Parliament.
Frauke Petry, who is considered a relative moderate within the highly fractious AfD, announced at the start of a Berlin news conference that the party had become "anarchical" and that it "cannot offer the voter a credible platform for government."
She then marched from the room without taking questions, leaving her AfD colleagues on the dais sputtering.
"I'd like to apologize in the name of my party," said AfD co-chairman Jörg Meuthen after Petry had left the room. "This wasn't discussed with us."
McAuley reported from Paris and Birnbaum from Brussels. Isaac Stanley-Becker in Berlin and Annabell Van den Berghe in Brussels contributed to this report.