On Monday, we published results from our recent poll of District residents finding that a solid majority, six in 10 residents, opposed a plan backed by Mayor Vincent C. Gray to build a stadium for the city’s pro soccer team, D.C. United. Those findings were met with disappointment, frustration and anger in some quarters — particularly among Gray aides and D.C. United fans, who felt the poll question’s wording was unfair and biased against the proposed deal.
City Administrator Allen Y. Lew wrote a letter to The Post, published Thursday, taking exception to the question’s phrasing — which referred to “using city funds to help finance a new soccer stadium” — saying the proposed deal does not use city funds to “finance” a stadium in any meaningful way. Had residents known more details of the deal, Lew said, they would have felt differently about it.
We welcome criticism and opportunities to improve the quality of our research, which is why we publish the full wording and order of all Post polls. In this case, we believe the poll’s survey question is fair in testing residents’ support for the city to offer financial help to a new soccer stadium. This, of course, is what the question asks:
While the degree and type of financial help the city provides has yet to be determined, we believe proposals to spend up to $150 million of city funds to purchase land, then leasing it to the team at $1 per year certainly qualifies as the financial help encompassed by the question.
Quality survey research on policy balances the need to ask clear questions that capture the core of a proposal with the need to track support over time. The soccer stadium question was a repeat from a 2008 survey, which found 36 percent favored and 60 percent opposed the use of city funds to help finance a soccer stadium. The fact that current results are nearly identical — 35 percent favor and 59 percent oppose — provides strong evidence that attitudes on this broad issue are stable.
Because the same broad language was used when The Post queried about a baseball stadium in 2002, finding 48 percent in favor, 47 percent opposed, it’s possible to say support for committing city funds to help build a soccer stadium is substantially lower than it was for a baseball stadium. This lower support is sensible, given that only 2 percent of D.C. sports fans in a 2011 Post poll said soccer was their favorite sport to watch, and 19 percent said they cared “a great deal” or “somewhat” about professional soccer.
Also note, in order to capture the public’s opinion of the soccer stadium without coloring their response with impressions of the baseball deal, we asked the soccer stadium question before, not after, we asked about the Nats stadium.
Sometimes more specific questions can produce differing results, though providing many details also runs the risk of confusing respondents or introducing bias. If we were to craft a very specific question about the proposed deal in a balanced way, we might ask —
There is a proposal to use up to $150 million in city funds to buy land where a new stadium can be built for the District’s Major League Soccer team, D.C. United. The city would maintain ownership of the property and rent it to the team for $1 per year, while the soccer team would pay to construct the actual stadium. Do you support or oppose this proposal?
This question illustrates some drawbacks of asking a balanced question that is more specifically tailored to a policy proposal. We don’t think this is a clearly better approach or would necessarily produce higher or lower support for the city’s plans, though the impact is unknown without additional research.
For example, a wording experiment embedded in our January poll question asking whether the Nationals stadium was a “good” or “bad” investment for the city produced strikingly small differences. A random half-sample of respondents was informed of the stadium’s estimated $650 million cost to the city, while the other half was not. But the percentage of respondents in each half-sample saying the stadium was a “good” investment differed only by a single percentage point: 71 percent among those informed of the cost, 72 percent among those who were not.
Differences in results due to question wording are possible and often unpredictable, which makes speculation about how a slightly different question might have produced much different results dicey without randomized experimentation. Given the lopsided nature of attitudes toward financially aiding a soccer stadium, though, differences in the question would need to be substantial to alter conclusions about support for the city proposal.