A few hundred men took to the streets in a suburb of this city early this month, furiously chanting for the expulsion of neighbors they described as interlopers.

The objects of their ire were Afghan refugees, millions of whom reside here in Pakistan. They are hardly newcomers — many fled war, Russian occupation or Taliban rule years or even decades ago. Many were born in Pakistan.

But the recent demonstration was a sign of bubbling discontent about Afghans in Pakistan, who comprise the world’s largest refugee population. While their presence has long been a source of tension, Pakistani politicians and the media are increasingly exaggerating their numbers and identifying them as a problem that must be solved as the neighboring nations eye the finale of the U.S.-led Afghan war, remote as that seems for now.

On an official visit to Australia last month, Pakistani Prime Minister Yousuf Raza Gilani called on the international community to help repatriate Afghans, who he said were “causing numerous difficulties” and spreading polio. In a recent interview, Interior Minister Rehman Malik accused the refugees of being “involved in criminal activities,” and said sending Afghans home was among Pakistan’s priorities.

The spotlight on Afghan refugees comes as the ever-wary neighbors trade barbs about cross-border violence and a potential negotiated settlement to the war in Afghanistan. Afghan officials, like their U.S. counterparts, have blamed Pakistan for fueling the Taliban insurgency, a claim Pakistan denies. But Pakistan wants a key role in reconciliation, and the refugees — who by most accounts Pakistan has hosted fairly graciously — could provide leverage.

At the same time, persistent violence has led to a decrease in refugee returns to Afghanistan, and there is scant sign that those remaining will soon leave. Amid a failing economy and political jockeying ahead of 2013 elections in Pakistan, analysts say Afghans are convenient targets. Indeed, the argument here echoes the U.S. immigration debate, with concerns about foreigners who commit crimes, steal jobs and fail to assimilate.

“We have been treating them as our brothers,” said Sher Bahadur, 64, one Nowshera resident who joined the recent demonstration, which took place after a fight between Pakistanis and Afghans. “Now the situation is so bad that we fear they have the might, power and resources to displace us.”

The complaints are not new, but the tenor has alarmed Afghan officials. One senior Afghan official, speaking on the condition of anonymity, said Pakistan is showing “early signs of new pressure” over refugees. The official said it was unclear whether the motivation is a desire to see Afghans leave, win additional refugee aid or blame Afghans for Taliban activity inside Pakistan.

According to the United Nations High Commissioner for Refugees, 1.7 million registered Afghan refugees live in Pakistan; the government says the figure is around 2 million. Another 1 million are believed to be in Pakistan illegally, said Habibullah Khan, secretary of the government’s States and Frontiers Regions Division. In the first 10 months of 2011, 43,000 Afghan refugees returned home, a figure that was 59 percent lower than the same period last year, the UNHCR said.

The majority of refugees are ethnic Pashtuns who have blended into Pakistan’s Pashtun-dominant belt along the border, which has long been poorly patrolled and traversed by migrant populations, including militants. Afghanistan, in fact, does not recognize the border, nor do many Pashtuns.

Originally housed in camps, most refugees now live in regular neighborhoods, where some have become fixtures in the transportation, clothing and carpet industries. Most are poorly-paid laborers.

There is little doubt the Afghans’ presence has affected Pakistan’s weak economy, but just how is debatable. Pakistan hosts more refugees for every dollar of per capita income than any other nation, which makes it difficult to absorb and support them, according to the UNHCR. But Afghans also contribute, said Rustam Shah Mohmand, Pakistan’s former refugee commissioner.

“Pakistan gets foreign exchange” from Afghan carpet exporters, Mohmand said. “Many have relatives in the West who send remittances. . . . They provide cheap farm labor to the landowners in the frontier.”

Yet many Pakistanis depict Afghans as drug- and gun-runners, in part because they are often arrested after militant attacks and violent crimes. The accusations are unfair, human rights advocates say.

Last year, the Pakistani government decided that all Afghan refugees would be “voluntarily” repatriated after the end of 2012. What that means remains unclear. A plan to offer visas will probably apply to only about 150,000 refugees, Khan said. But Tim Irwin, a UNHCR spokesman, said “there’s certainly no talk of anyone being forced back.”

There is such talk in Nowshera, however, where thousands of Afghans live. The recent fight broke out with a quarrel between Afghan and Pakistani youths, after which adults jumped into the fray, residents said. Pakistanis — who refer to themselves as “locals” — said Afghans attacked with rods, wounding several, then followed them to the hospital with Kalashnikovs.

Last week, dozens of Pakistani men packed into one elder’s home and recited grievances: Afghans keep to themselves, and they insulted Pakistan during the brawl. They are rich and buy off police. They are bad drivers.

“We are Pashtun, but we are not Afghan. We are Pakistani,” said Mohammed Akbar, 31. A man sitting on a sofa interjected: “The Afghans should go back!”

Yet a visit to Afghan elders — at the grand home of a clothing importer — revealed how indelibly the immigrants have become part of the landscape. Several had lived in Pakistan for 40 years and held dual citizenship. The fight, they shrugged, was a mere scuffle being exploited by Pakistani community leaders for political gain.

“We are mixing. But whenever such an incident happens, they label us Kabulis,” or Kabul natives, said refugee Jamil Khan, 23, who participated in the fight.

Both sides said the issue would be settled by elders, according to local tradition. But the Pakistanis said tensions would remain rife.

“Nobody believes that they will go,” said Liaqat Gilani, a former district mayor.

A short drive away at a former Afghan refugee camp that is now a squalid slum, truck owner Watan Khan, 39, said he has no plan to return to the home town he left in 1978, in Afghanistan’s Taliban-riddled Logar province. Therefore, he said, he has no right to complain about Pakistani treatment.

“Even if our lives are not as good as locals, we have no choice,” Khan said. “We are living in someone else’s land.”