Hawaii: Where good polling goes to die


Sen. Brian Schatz and challenger Rep. Colleen Hanabusa before their July 17 debate ahead of the Democratic Senate primary. (Cindy Ellen Russell/the Star-Advertiser via Associated Press)

Hawaii's governor and one of its two senators could lose their primaries on Saturday, with a new poll showing both Gov. Neil Abercrombie (D) and appointed Sen. Brian Schatz (D) falling behind their challengers. The Honolulu Star-Advertiser poll, conducted by Ward Research, shows Abercrombie trailing state Sen. David Ige by 18 points and Schatz trailing Rep. Colleen Hanabusa by eight.

Pretty dire straits, right?

One problem: Polling in Hawaii — and especially primary polling — is notoriously unreliable. And we don't mean off-by-a-few points unreliable. We mean often vastly different-from-the-final-result unreliable.

In fact, in the Senate race, another recent poll had showed Schatz with a clear lead. The automated poll for the Civil Beat Web site, by the Merriman River Group, showed Schatz ahead by the same margin by which the Star-Advertiser shows Hanabusa leading — eight points.

So whom to believe? Probably nobody — at least insofar as you're trying to predict what happens Saturday. The recent history of Hawaii polling is not a happy one, with both of these pollsters showing late surveys whose results were far afield of the eventual margin.

Here's a recap of some major recent Democratic primaries, along with one special election (the 2010 race for Hawaii's 1st District). Results from late Civil Beat and Star-Advertiser polls — from the final few weeks of the campaign — are compared with the actual margin, with polls showing the wrong winner in the negative:

(A note on the above: Civil Beat wasn't around to poll the 2002 election, so its results aren't included. Also, the Star-Advertiser merger happened in 2010. Previous polls were conducted for the Advertiser, but given that the pollster is the same, they are listed above as Star-Advertiser polls.)

Civil Beat missed the final margin in the 2012 Democratic Senate primary by 18 points and the 2010 special election by six points. In the former case, it had former congressman Ed Case (D-Hawaii) leading then-Rep. Mazie Hirono by one point; Hirono won a little more than a week later by 17 points.

Star-Advertiser/Advertiser polling differed from the final result in the Democratic primary in the state's 2012 2nd Congressional District by 29 points(!) and in the state's 2002 governor primary by 19 points. In the latter case, it was again Case vs. Hirono. The poll had Hirono up by 20 points; two weeks later, she won by one.

This is not meant to pick on pollsters. Polls, after all, are not meant to predict elections, and things can theoretically change in a hurry in the final weeks of an election.

Also, as we've written before, polling in Hawaii is difficult for a host of reasons — and in large part because the state's large Asian American population is more difficult to correctly sample and often less willing to answer a pollster's questions.

This is especially a problem given that many of the state's big Democratic primaries pit a white candidate against an Asian American one. Advertiser polling in 2002 and 2010 showed a better picture for the Asian American candidates in those races than was probably the case, while Civil Beat's 2012 poll skewed heavily toward the white candidate, Case.

Both the state's gubernatorial and Senate races this year, for what it's worth, again pit a white candidate against an Asian American one.

Abercrombie is definitely in trouble — both Civil Beat and Star-Advertiser polling show this — but the Schatz-Hanabusa race shouldn't be judged by one poll.

Ultimately, the best course of action when it comes to polls in Hawaii is to take them with a grain of salt. They have been off before, and on Saturday, one of them is liable to be off again.

Update 1:20 p.m.: Civil Beat just happens to have posted an explanation today about why it's 2012 polling was off and what it has done since to address the issue (h/t Seth Rosenthal):

Looking closely at the those polls, it quickly became apparent that the results had systematically underestimated Senator Hirono’s standing among AJA voters — our polls frequently found her to be receiving just about half of the AJA vote, which we concluded was at odds with our knowledge of Hawaii’s political landscape, not to mention common sense. And given that the Senate primary pitted an AJA candidate against a Caucasian, this mattered a great deal to the polling outcomes (in contrast to the CD 2 race, where voter ethnicity played very little role).

To figure out how this happened, and guard against it happening again, we dedicated a portion of each of our Civil Beat polls in 2013 and early 2014 to testing a number of hypotheses. In fact, if you have responded to any of our polls during the past two years, you may have been asked to answer a few questions about how you voted in 2012. Those questions were included to help us resolve the issues we encountered in 2012.

More here.

Aaron Blake covers national politics and writes regularly for The Fix.
SECTION: {section=politics, subsection=null}!!!
INITIAL commentConfig: {includereply=true, canvas_permalink_id=washpost.com/8bvh5zpd9k, allow_comments=true, commentmaxlength=2000, includeshare=true, display_comments=true, canvas_permalink_app_instance=m6yzjj840m, display_more=true, moderationrequired=false, includefeaturenotification=true, canvas_allcomments_id=washpost.com/km4ey0dajm, comments_period=14, defaultsort=reverseChronological, includevoteofftopic=false, allow_videos=false, childrenitemsperpage=3, markerdisplay=post_commenter:Post Commenter|staff:Post Writer|top_commenter:Post Forum|top_local:Washingtologist|top_sports:SuperFan|fact_checker:Fact Checker|post_recommended:Post Recommended|world_watcher:World Watcher|cultuer_connoisseur:Culture Connoisseur|weather_watcher:Capital Weather Watcher|post_contributor:Post Contributor, includesorts=true, includeheader=true, defaulttab=all, includeverifiedcommenters=true, includerecommend=true, maxitemstop=2, includereport=true, source=washpost.com, allow_photos=false, maxitems=7, display_ugc_photos=false, includepause=true, canvas_allcomments_app_instance=6634zxcgfd, includepermalink=false}!!!

UGC FROM ARTICLE: !!!

FINAL commentConfig: {includereply=true, canvas_permalink_id=washpost.com/8bvh5zpd9k, allow_comments=true, commentmaxlength=2000, includeshare=true, display_comments=true, canvas_permalink_app_instance=m6yzjj840m, display_more=true, moderationrequired=false, includefeaturenotification=true, canvas_allcomments_id=washpost.com/km4ey0dajm, comments_period=14, defaultsort=reverseChronological, includevoteofftopic=false, allow_videos=false, childrenitemsperpage=3, markerdisplay=post_commenter:Post Commenter|staff:Post Writer|top_commenter:Post Forum|top_local:Washingtologist|top_sports:SuperFan|fact_checker:Fact Checker|post_recommended:Post Recommended|world_watcher:World Watcher|cultuer_connoisseur:Culture Connoisseur|weather_watcher:Capital Weather Watcher|post_contributor:Post Contributor, includesorts=true, includeheader=true, defaulttab=all, includeverifiedcommenters=true, includerecommend=true, maxitemstop=2, includereport=true, source=washpost.com, allow_photos=false, maxitems=7, display_ugc_photos=false, includepause=true, canvas_allcomments_app_instance=6634zxcgfd, includepermalink=false}!!
Comments
SECTION: {section=politics, subsection=null}!!!
INITIAL commentConfig: {includereply=true, canvas_permalink_id=washpost.com/8bvh5zpd9k, allow_comments=true, commentmaxlength=2000, includeshare=true, display_comments=true, canvas_permalink_app_instance=m6yzjj840m, display_more=true, moderationrequired=false, includefeaturenotification=true, canvas_allcomments_id=washpost.com/km4ey0dajm, comments_period=14, defaultsort=reverseChronological, includevoteofftopic=false, allow_videos=false, childrenitemsperpage=3, markerdisplay=post_commenter:Post Commenter|staff:Post Writer|top_commenter:Post Forum|top_local:Washingtologist|top_sports:SuperFan|fact_checker:Fact Checker|post_recommended:Post Recommended|world_watcher:World Watcher|cultuer_connoisseur:Culture Connoisseur|weather_watcher:Capital Weather Watcher|post_contributor:Post Contributor, includesorts=true, includeheader=true, defaulttab=all, includeverifiedcommenters=true, includerecommend=true, maxitemstop=2, includereport=true, source=washpost.com, allow_photos=false, maxitems=7, display_ugc_photos=false, includepause=true, canvas_allcomments_app_instance=6634zxcgfd, includepermalink=false}!!!

UGC FROM ARTICLE: !!!

FINAL commentConfig: {includereply=true, canvas_permalink_id=washpost.com/8bvh5zpd9k, allow_comments=true, commentmaxlength=2000, includeshare=true, display_comments=true, canvas_permalink_app_instance=m6yzjj840m, display_more=true, moderationrequired=false, includefeaturenotification=true, canvas_allcomments_id=washpost.com/km4ey0dajm, comments_period=14, defaultsort=reverseChronological, includevoteofftopic=false, allow_videos=false, childrenitemsperpage=3, markerdisplay=post_commenter:Post Commenter|staff:Post Writer|top_commenter:Post Forum|top_local:Washingtologist|top_sports:SuperFan|fact_checker:Fact Checker|post_recommended:Post Recommended|world_watcher:World Watcher|cultuer_connoisseur:Culture Connoisseur|weather_watcher:Capital Weather Watcher|post_contributor:Post Contributor, includesorts=true, includeheader=true, defaulttab=all, includeverifiedcommenters=true, includerecommend=true, maxitemstop=2, includereport=true, source=washpost.com, allow_photos=false, maxitems=7, display_ugc_photos=false, includepause=true, canvas_allcomments_app_instance=6634zxcgfd, includepermalink=false}!!
Show Comments
Next Story
Philip Bump · August 4