President Biden signaled Thursday that he will agree to leave some U.S. forces in Afghanistan past a May 1 deadline for withdrawal, but said he “can’t picture” U.S. troops still on the job there next year.

Biden cited the logistical challenge of quickly removing U.S. and allied forces this spring but indicated that he contemplated only a short delay in ending the deployment of American battlefield forces after nearly two decades of war.

“We will leave, but the question is when we leave,” Biden said during a news conference at the White House.

Asked whether he expected U.S. forces to still be there next year, Biden replied that he “can’t picture that being the case.”

The administration had already signaled that it would probably miss the deadline set by the Trump administration during historic negotiations with the Taliban insurgent group. The planned withdrawal has been on hold while the Biden administration examined terms of the 2020 agreement and considered whether to fully honor the deal.

“The answer is that it’s going to be hard to meet the May 1 deadline; just in terms of tactical reasons, it’s hard to get those troops out,” Biden said in response to a question about the deadline. “If we leave, we’re going to do so in a safe and orderly way,” and in cooperation with allies that also have forces in Afghanistan, Biden said.

Biden did not address whether he wants to replace the U.S. military deployment with a small contingent of counterterrorism forces as has been recommended by some military leaders. To do so would violate the Taliban’s agreement with the United States, but a delay in withdrawing existing forces could provide a window to negotiate.

The Biden administration is seeking a power-sharing deal between the Taliban and the elected government in Kabul.

The Taliban has demanded the withdrawal of all foreign forces and warned of a “reaction” if Biden does not fulfill the May 1 agreement.

About 2,500 U.S. troops remain in Afghanistan, according to the Pentagon, with several hundred more deployed on a short-term basis.

At the height of the war in 2010, the United States had more than 100,000 troops spread across the country, many in combat daily. More than 2,300 U.S. troops have been killed in Afghanistan since U.S. forces drove the Taliban from power in a war launched after al-Qaeda’s September 2001 terrorist attacks on the United States.

Rep. Adam Smith (D-Wash.), the chairman of the House Armed Services Committee, said Wednesday that the administration wanted to explore a new understanding with the insurgent group that would allow the counterterror force to stay. Smith also said during a “Foreign Policy” magazine forum that such an agreement may be impossible.

As a presidential candidate, Biden pledged to end America’s longest war and wrote that it was past time for U.S. forces to depart. Secretary of State Antony Blinken has said that the United States would probably not start what became the Afghan war if faced with the same choices today, but he like other U.S. officials has also said that the country cannot be allowed to again become a lawless haven for terrorists.

Donald Trump came closer to ending the war than his predecessors, arguing that U.S. forces were being used as little more than police officers and construction crews.

Biden appeared to criticize the peace agreement Trump oversaw, which was brokered over objections from the U.S.-backed government in Kabul, but not the underlying decision to end the U.S. military deployment.

“It is not my intention to stay there for a long time. But the question is how and on what circumstances do we meet that agreement that was made by President Trump to leave under a deal that looks like it’s not being able to be worked out to begin with?” Biden said. “How’s that done? But we are not staying a long time.”

Turkey announced last week that it will host a peace summit in April that was requested by the Biden administration in an effort to jump-start negotiations between the Afghan government and the Taliban.

The deal the Trump administration negotiated last year did not require the Taliban to reach a peace accord with the Afghan government first, angering the elected civilian government.

Defense Secretary Lloyd Austin visited Afghanistan last weekend, becoming the first senior Biden administration official to do so.

Austin told reporters traveling with him in Kabul that senior U.S. officials want to see “a responsible end to this conflict” and “a transition to something else” after nearly 20 years of war.

“There’s always going to be concerns about things one way or the other, but I think there is a lot of energy focused on doing what is necessary to bring about a responsible end and a negotiated settlement to the war,” Austin said Sunday.