The Federal Communications Commission said on Tuesday that more than half a million public comments were, in fact, missing from the bulk download that the agency had shared with the outside world in a bid to add some measure of transparency to the debate over net neutrality.
The counting and miscounting of those those public submissions has been the subject of a fierce back-and-forth between those supporting and opposing the passage of rules that would govern how equally online content must be treated by Internet service providers.
According to a blog post by FCC special counsel for external affairs Gigi Sohn and chief information officer David Bray, there were indeed roughly 4 million comments submitted to the FCC on its open Internet rulemaking. That's the number cited by President Obama in his call for the strongest possible net neutrality rules.
But when the FCC turned that comment cache into a file available for the public to download from the Internet -- sort of like turning over a collection of the votes cast in an election -- some 680,000 comments were lost in the conversion.
The FCC offered that bulk download of the commenting record to try to recruit the public's help in making sense of the historic number of comments that flooded in. And in that way, the system worked especially well. Advocacy groups and analysts began parsing the data given to them in XML format in an attempt to understand what the public was saying about net neutrality issue. The discrepancies found during that process forced the FCC to double-check its work. (The FCC will repost a new, complete XML file after the holidays, the agency says.)
It should be noted that, according to the FCC, those comments were never missing from the files that its regulators use internally and that are available to the public one-by-one on FCC.gov. But that external record has become important fodder for the public policy debate over net neutrality.
The FCC has also gone back and clarified what it says are fine-grained numbers it misreported earlier on its blog. The number of comments that came in during the second, "reply comments," round of the process was not 2,445,000 but 2,552,000. And, the agency said, there were more comments than reported that came through the agency's online commenting system and a supplementary bulk uploading tool, and fewer than initially reported that came through the e-mail address that the FCC set up to help handle the influx of public comments.
In this, the age of data, how did the situation get quite so messy? In large part, it's due to a Web-based system for capturing comments that was built during the Clinton administration and hardly updated since.
"We think it’s important that people understand that much of the confusion stems from the fact that the Commission has an 18-year-old Electronic Comment Filing system (ECFS), which was not built to handle this unprecedented volume of comments nor initially designed to export comments via XML," write Sohn and Bray. "This forced the Commission’s information technology team to cobble solutions together MacGyver-style."
In the hopes of making such dramatic measures unnecessary in the future, the FCC has asked Congress for more funding to upgrade the Electronic Comment Filing System. It has not yet received it.