Texas Gov. Rick Perry (R) on Thursday talked about a “very real possibility” that Islamic State extremists may have crossed the U.S.-Mexico border, though he added there is “no clear evidence of that.” Were his remarks alarmist?

To be clear, Perry’s assessment was not unprompted. He was responding to an audience member’s question about that specific homeland security threat during an event hosted by the conservative Heritage Foundation.

Nonetheless, the governor’s comments are sure to stoke serious concerns about domestic security in light of the recent atrocities committed by the Islamic State (often known as ISIS or ISIL), not to mention the organization’s stated goal of harming Americans, and Defense Secretary Chuck Hagel’s description of the terrorist outfit as an “apocalyptic” group “beyond anything we have seen before.”

Yikes. No one wants any of that creeping through our borders, but there appears to be no sign of that happening yet. Rear Adm. John Kirby, the Pentagon’s press secretary, discussed the issue during an appearance Friday on CNN’s “New Day.”

“I’ve seen no indication that they are coming across the border with Mexico,” Kirby said. “We have no information that leads us to believe that. That said, we do know they have aspirations to hit western targets and it’s something, as Secretary Hagel said yesterday, that we’ve got to take seriously and we have to try to be ready for it.”

Host Kate Bolduan then asked whether the administration has an estimate of how many individuals with U.S. passports sympathize with the group and pose a threat to Americans.

“I don’t know that we have a hard and fast number,” Kirby said. “But you bring a really interesting point to light, and that is the growth of foreign fighters inside [Islamic State]. I know the British are now trying to investigate to see who this murderer was who killed Jim Foley [the U.S. journalist beheaded by the group] because of his British accent.”

Kirby also noted that the Australian government recently acknowledged that some of its citizens had become radicalized and joined the Islamic State. “It’s a problem in many countries, and we face that problem here in America,” he said. ” It’s just hard to get our hands around it.”

Based on what Kirby said, there is plenty of reason for U.S. national-security agencies to show increased vigilance against the threats, but there is no clear reason for Americans to panic at this point.

The Department of Homeland Security did not immediately respond to questions this afternoon about whether the Obama administration knows of Islamic State elements sneaking through the U.S.-Mexico border.