First, the good parts in Obama’s NSA reform speech: He outlined a number of reforms that will increase transparency into collection of metadata and the legal rationale for it. He called for the creation of a panel of outsiders that will represent the public before the FISA court, meaning that it won’t be hearing only the government’s arguments. He said a court order will now be required before the government can get access to that data.

Obama repeatedly said that the public’s confidence that their privacy is being safeguarded is a paramount goal. He recognized that civil libertarians are right in seeing the potential for abuse in the bulk collection of meta-data. And he announced an end to such telephone meta-data collection — at least by the government itself.

But this last point drives home that there remains one major unanswered question — one that will continue to be central to this whole debate: What will future telephone metadata collection look like? Obama ended the government’s direct collection of it, and said he’s directed the Attorney General to come up with a substitute method. Members of Congress will also be consulted.

In this sense, while Obama’s speech went some way towards addressing concerns held by civil libertarians, it also confirmed that his core difference with them remains. Civil libertarians don’t believe that bulk collection of telephone metadata should be required by the government — no matter who collects it. This isn’t to say they oppose government access to metadata on principle, provided a court green-lights it and provided phone companies had voluntarily kept that data. It’s the government-mandated collection of this data that’s the problem.

“Anytime you assemble a massive database of sensitive information, you make everyone less secure,” Alex Abdo, an attorney with the ACLU, argues. “We are fundamentally less secure when we have our sensitive information aggregated, whether by government or by companies forced to keep it for the government. Until further notice, bulk surveillance is in place.”

The basic situation is this: Obama did not back away from the core principle that this bulk collection is necessary to keep Americans safe. As he put it, the goal will be to “establish a mechanism that preserves the capabilities we need without the government holding this bulk metadata.” The key word there being “preserve.”

This isn’t to say the reforms Obama did outline aren’t significant. “Make no mistake, this is a major milestone in our longstanding efforts to reform the National Security Agency’s bulk collection program,” Senator Ron Wyden, a fierce critic of NSA overreach, said in a statement. Wyden added that he will push for further reforms, and noted that today’s speech was “vindication” for public “activism” against that overreach. Crucially, Wyden noted that this activism will continue.

In short: the debate isn’t over. Obama’s announcement itself — with its call for Congressional consultation — ensures this.

And that debate will probably continue revolving around this: there probably isn’t any solution to the core question over bulk collection that can make both sides happy. For Obama, the debate should be about how to collect this metadata in a way that maximizes public confidence that privacy rights are protected. For civil libertarians, the debate should be over whether to collect that metadata at all — the bulk collection itself is an abuse, one that is not central to keeping the nation safe, despite Obama’s protestation otherwise. Indeed, bulk collection, in this telling, is itself a continued threat to people’s security.

It may well be that this debate can only be settled in the court of public opinion — not to mention in the courts themselves.