Within that is the "flight restricted zone," which offers a tighter set of constraints, allowing government and commercial aircraft (allowing the latter to land at Reagan National). (Incidentally, the outlines of these areas come via the very handy DC geography atlas.)
Update: Steve Hedges of the Aircraft Owners and Pilots Association notes that there are three general aviation airports in the FRZ (Potomac, Hyde and College Park) which allow private pilots to operate.
And then there's the "P56 Zone."
We wrote about this a bit when the drone crashed on the grounds of the White House. But it's an area in which no flying is allowed at all beneath 18,000 feet. It covers the Mall, the White House, and the Capitol -- as well as the Naval Observatory in northwest D.C.
Where the guy crashed his gyrocopter, then, was in a very restricted area within a restricted area within a limited area. It is perhaps one of the places in the United States that you are most not allowed to fly.
Which explains why he was taken into custody.
Update: Here is precisely where the gyrocopter landed, courtesy of the Post's graphics department: