President Obama meets with Prime Minister of India Narendra Modi in the Oval Office on Tuesday. (Dennis Brack / Pool/EPA)

Indian Prime Minister Narendra Modi will address a joint meeting of Congress on Wednesday, a rare honor that contrasts sharply with the affront he faced more than a decade ago when he was denied a U.S. visa after Hindu riots killed more than 1,000 people in the state he then governed.

But even as many Indian Americans cheer the ascent of Modi, who met Tuesday with President Obama at the White House, others say they continue to be troubled by his record on human rights since he became prime minister in 2014.

As he speaks at the Capitol, a group of Indian American Christians will hold a prayer vigil a few blocks away to protest what they call a pattern of abuse against religious minorities in India, including the torching of churches and the lynching of Muslims for eating beef.

“We welcomed Mr. Modi when he was elected, but he has driven the country into religious polarization,” said John Prabhudoss, president of the Federation of Indian American Christian Organizations. “He took an oath to protect the constitution and all the people, but he has failed to uphold it.”

Modi, 65, is a leader of India’s Hindu nationalist party and its pro-Hindu movement. Supporters say he has worked hard to dispell his image as a religious fanatic, stressing an agenda of economic development and improved ties with Washington.

“We are building strong ties between India and the U.S., and we have to look to the future,” said Vic Chauhan, a defense consultant in Northern Virginia. He described his and Modi’s home state of Gujarat as peaceful and secure, and he said it was unfair to blame Modi for the 2002 riots. “If we keep going back to historic issues, we will never move ahead.”

Still, the episode keeps returning to haunt Modi. Last week, a judge convicted 24 people of involvement in killing Muslims during the two-month spate of violence in 2002, although he acquitted an official of Modi’s Bharatiya Janata Party.

The prime minister, accused of failing to quell the Gujarat riots as governor, was previously cleared of blame by India’s supreme court. But some minority groups fear his rise to national power has emboldened rightist Hindu groups to harrass minorities.

A recent report by the nonprofit U.S. Committee on International Religious Freedom found that “incidents of religiously-motivated and communal violence reportedly have increased for three consecutive years,” triggered by divisive campaigning that led up to Modi’s election as prime minister.

According to Prabhudoss, Hindu groups have invaded and vandalized churches, accusing priests and pastors of forcibly converting impoverished Indians to Christianity. In some cases, he said, “elected and party officials are leading the mobs.”

Muslims, a much larger minority group than Christians in India, have also faced increasing hostility. Hindus regard cows as sacred and believe beef should not be eaten, and Modi’s party has been pushing states to pass laws banning beef. Last year, a Muslim was beaten to death on suspicion of stealing a calf, and another was lynched for allegedly storing beef.

Kaleem Kawaja, an official of the Indian American Muslim Council, said Modi has made many inflammatory speeches that incite Hindus against Muslims, especially those coming from neighboring Bangladesh.

“It is just like Donald Trump talking about Mexicans,” Kawaja said. “Modi uses the same language and inflames passions. This aggravates Hindus against Muslims, and it can do great damage to our secular democracy.”

For many Indian Americans, such concerns are outweighed by Modi’s efforts to bring new stature to their homeland and forge closer ties with Washington, which once shunned India for its nonaligned stance in the Cold War and for conducting nuclear tests.

The prime minister has been eagerly embraced by Indian American business groups, which will host a reception on Capitol Hill after his speech.

The majority of the estimated 3 million Indians living in this country are Hindu, and many are from Gujarat. But Modi’s popularity is not limited to Hindus. One fan is Ajay Kothari, an engineer and federal contractor in suburban Maryland. Kothari is a Jain, another minority religion that focuses on kindness to all living creatures.

“Whatever happened in 2002, I hope it doesn’t happen again, but by and large I feel Muslims and Hindus and other groups in India have lived quite well together,” Kothari said.

Modi, he said, has done something important for all Indians. “He has brought back a sense of pride and identity that was gone.”