Afghan opposition leader Abdullah Abdullah was at the rally when the attack began but escaped unhurt, according to his spokesman.
The attack comes just days after the signing of a peace deal between the United States and the Taliban and highlights the likelihood of continued violence in Afghanistan despite peace talks.
The Islamic State is just one of many armed groups in the country that oppose the current peace efforts. The Islamic State is neither allied with the Afghan government nor the Taliban, and it endorses attacks on civilians who do not adhere to the group's fundamentalist interpretation of Islam.
The Taliban quickly issued a statement Friday saying it was not behind the attack. The United States and the Taliban signed a peace deal Saturday, and since then, those militants have launched attacks on Afghan government forces but are under pressure to minimize civilian casualties.
The attack in southwestern Kabul began when two gunmen in a nearby multistory building fired down into a crowd paying tribute to a revered Shiite martyr. Armed with machine guns, hand grenades and rocket-propelled grenades, the attackers quickly caused massive bloodshed and held off a unit of elite Afghan police for hours more, according to Interior Ministry spokesman Nasrat Rahimi.
Hayatullah Asadi ran for cover behind a wall when the shooting began. "People were terrified, they were screaming and fleeing," he told The Washington Post from a nearby hospital where many of the wounded were being treated.
The last attacker was killed more than five hours after the initial gunfire, according to a senior Afghan official who spoke on the condition of anonymity because he was not authorized to speak to the media while the investigation into the attack was ongoing. The official said the large number of interconnected rooms and the presence of civilians in the building used by the attackers slowed efforts to reach them.
Rahimi, the Interior Ministry spokesman, said 32 people were killed and 81 wounded Friday. The casualties included women and children.
The U.S. military command in Kabul said the response to Friday's attack was "Afghan-led and Afghan executed, with advice and assistance from U.S. forces, to include medical assistance to those injured," according to a statement sent to The Washington Post.
The attack also comes at a time of deep political divisions in Afghanistan. Afghan President Ashraf Ghani and his rival Abdullah both declared victory when election results were announced last month.
Both Ghani and Abdullah condemned the attack Friday. Ghani also called Abdullah on Friday to assure him of his "well-being" afterward, according to a tweet from his office.
The two men are also at odds over how to proceed with peace talks with the Taliban. The U.S.-Taliban peace deal signed Saturday set a March 10 deadline for the start of intra-Afghan talks. But there is a dispute over whether the government should release thousands of Taliban prisoners ahead of those talks or use them as leverage in the negotiations.
The Afghan capital has enjoyed a period of relative calm in recent months as U.S. negotiators pressed the Taliban to halt large-scale attacks there in the lead-up to the signing of the peace deal.
Additionally, military pressure on the Islamic State in Afghanistan brought with it a drop in that group’s attacks nationwide. The last large-scale attack in Kabul was claimed by the group in August, when a suicide bomber targeted a wedding hall, killing 63 people, mostly members of the ethnic Hazara and Shiite minority.
Sharif Hassan contributed to this report.