There’s a fascinating discussion going on at the Monkey Cage and elsewhere about what factors actually help determine the results of U.S. elections. Many political scientists have argued that the state of the economy plays a huge, often decisive role in presidential elections (here’s Nate Silver’s exhaustive attempt to figure out just what specific economic variables have mattered). But what about campaigns themselves? Do all those ads and gaffes and millions of dollars in spending have an impact at all?

Does any of this even matter? (Joe Michaud-Scorza/AP)

On the other hand, if both candidates in a race are running the best campaigns they possibly can (and both parties are typically putting up the strongest possible candidates), then it’s possible the campaign effects just balance out. That doesn’t mean campaign strategy is meaningless — just that its impact can be hard to detect from the data. James Fearon says we’d need randomized controlled trials to really figure out what works and what doesn’t in political campaigns:

The experiment we’d like to run is to convince candidates to randomly choose from a menu of different campaign strategies, and do this for lots of elections so we could identify average effects. Not going to happen for U.S. presidential campaigns. (Though Leonard Wantchekon famously pulled it off with candidates for a parliamentary election in Benin, and I’ll bet we start seeing some of this for lower level elections in the U.S. in the coming years.)

Here’s an old New York Times story on Leonard Wantchekon’s experiments with candidates in Benin. Among other things, he found, using randomized control trials, that incumbents do best when they make specific promises, such as pledging to build a new school or road. Challengers, on the other hand, do better when they make broad but vague promises and appeals. So how many local candidates in the United States would want to sign up for something like this?