Architecturally cool new buildings for corporations may be all the rage. Apple is building a "spaceship" campus. Google has proposed futuristic buildings that include an "architectural sky." Amazon's new headquarters will include 100-foot-tall orbs known as "biospheres."

But one of the most innovative workspaces in history was an ugly, hastily designed-and-constructed wood-and-concrete structure on MIT's campus, now torn down, known as the "magical incubator": the legendary Building 20. Famous as the place that was home to nine Nobel Prize winners, that birthed the first commercial atomic clock and housed pioneering linguist Noam Chomsky, it worked not because of soaring ceilings or flooding daylight or pristine materials. It worked, journalist and economist Tim Harford argues in a new book, because it was messy, unorganized, reconfigurable, and under the control of the people who worked in it. "Nobody cared what happened there--if you wanted to knock out two floors," Harford said in an interview, "who cares? You just do it. Take out two floors. Everybody was thrown in there in a jumble."

It's just one example Harford uses in his latest book, Messy: The Power of Disorder to Transform Our Lives, to make his case for the under-appreciated role of disorganization, both in our personal and professional realms. Yet another addition to the pop-economics genre -- think Freakonomics, Predictably Irrational, and Harford's own The Undercover Economist -- the new book by the Financial Times columnist and visiting fellow at Oxford University goes beyond the well-worn discussion over messy desks as a sign of creativity. (Reading about it, however, never fails to reassure those of us who have one).

Rather, he tackles the notion more conceptually, examining why rigid targets can wreak havoc, how unpredictable leaders get ahead, and when flexibility and improvisation matter more. OnLeadership spoke with Harford about his book, our obsesssion with digital tidiness, the biggest scandal in the Volkswagen emissions story, and what he thinks of Donald Trump's campaign. The conversation below has been edited for length and clarity.

Did you get the idea to write this as an antidote to the obsession with [organizational consultant] Marie Kondo?

I suppose. But I’m actually a fan of Marie Kondo. One of the things Kondo says in her book, which I think is absolutely right, is you cannot declutter your house through organizational systems. They don’t work, and that is a point where we’re totally in agreement.

But I’m not just talking about physical mess. I’m talking about all kinds of conceptual messes and disorder and difficult people and diversity and unpromising circumstances and improvisation. My argument is that we often take organizational systems  — maybe they’re PowerPoint slides, maybe they’re targets and metrics, maybe they’re clean desks, all kinds of things — out of situations where they work really well and force them into situations where they don’t work at all. We just feel they should work, because the messier, less planned, less tidy approach — it just frightens us. It makes us feel anxious.

The idea that a messy desk is a sign of a creative mind is quite well known. What does your book adds to that discussion, on physical tidiness, particularly in the workplace?

We often have offices where some kind of clean desk policy is imposed on people, or in some way their physical environment is heavily constrained. We know that’s annoying. But the psychological research also says this is profoundly disempowering and frustrating. It damages productivity. It makes people feel physically uncomfortable. Very often, there’s no reason behind it. It’s just some guy somewhere who thinks that this is what professionalism looks like.

You’ve got a constant inflow of information -- digital information and physical information. You’ve basically got two overall strategies: Do you tidy all this stuff into filing cabinets and label everything? Or do you leave it on your desk? It turns out that usually leaving it on your desk is a better strategy. It looks disorganized. It looks messy. But your desk is actually organizing itself. The good stuff you’re touching rises to the top of the piles of paper and the stuff you’re not touching goes to the bottom.

What about digital information? Why has ‘Inbox Zero’ become such a holy grail?

When I left my desk I think [my inbox] had four emails. But what I don’t have is lots of folders. You can leave your email in your inbox or delete loads of stuff, put stuff you need to act on in a folder named action, and everything else you just archive. I think either works. It’s partly a matter of personality and partly a matter of habit. But what doesn’t work is this weird situation of, in trying to get your inbox to zero, you take your 17,000 emails and you start categorizing them and putting them into folders, which takes forever and makes precisely no difference in the likelihood you will find anything.

Is there research that backs that up?

There is very good research. There’s a guy named Steve Whittaker, he worked for a bunch of different tech companies before becoming an academic psychologist. What he did with an academic colleague was to install spyware, with permission, on a bunch of computers, so they could study nearly 100,000 attempts by people to find email. Did these people click to a folder and start scrolling? Or did they use the search bar? The paper is actually called ‘Am I wasting time organizing email?’ And the answer is yes. You compare these different methods, and basically the search bar works fine. It’s quicker. It’s just as likely to end in a success, and that doesn’t include any of the time you spent creating those folders. So it’s just not worth bothering with it unless you have something that has a really, really clear structure.

You devote a chapter to the rigidity of incentives. How do you reconcile the need for corporate goals with the power of disorder?

Actually, the Nobel Prize in economics was given this year for these issues. One of the two prize winners, Bengt Holmström, thought a lot about the theoretical basis of this. He fundamentally said if you’re paying somebody to do a complex job with lots of different elements in a team, and you can’t necessarily say who’s contributing what and you can’t measure some of the tasks, don’t use a strong incentive. Don’t use these aggressive targets, because you are basically paying people to lie to you or to game the system or to cheat or to stab each other in the back. You only want to do that if you really feel you can measure everything that matters.

It’s just ignored all the time, and I think the reason is because bosses see their organizations through this filter of the Excel spreadsheet and the PowerPoint slide. They see their financials and it can lead to a kind of state of mind where you go 'OK, if we could get that metric to move in a favorable direction then we’ll be fine.'

We of course have this classic recent example in Wells Fargo, where the metric [was based on] cross selling. If Wells Fargo can encourage people to open more accounts by being great at what Wells Fargo does, of course that’s going to be good for business. But the moment you come in with a heavy-handed incentive, you get the behavior we see, which is people pressurizing friends and families to open the accounts, and people secretly opening them on behalf of customers without telling them. 

What does that have to do with disorder?

You’re interpreting your organization through these nice simple metrics. It all looks so comprehensible and tidy on the spreadsheet or on graphs, on PowerPoint slides. Those little lines and graphs and dots: They seem so clear. But of course they are tidy summaries of a messy reality. The moment you try to impose the target, you find that reality bites back. You actually have to see what is going on on the ground, which is always more complicated and subject to human error and office politics. It’s much harder for a CEO to do that.

There’s all these new apps and ways to help us manage our time — some even use machine learning to help us fill our calendar. What does the research tell us about managing our calendars?

My instinct has been that you should only put things in the calendar that absolutely have to go there. Another point of view says you should fill the calendar with tasks and use your calendar as the to do list. The question is, who’s right? There’s some very interesting research done by three psychologists nearly 35 years ago. They split students into control groups and these various groups got different advice. Students who were advised to plan in detail, day by day -- the psychologists thought that would work best.

But in fact that was catastrophic. Students got very discouraged very quickly. What seemed to be happening is you block out your day, and then a friend comes for coffee. The washing machine breaks down. An unexpected phone call just gets in the way, and you’re immediately running behind. So now what do you do? These students became very discouraged and they stopped working. Their study time was way down.

On the other hand, the students who were told to just set out their goals for the month did way better. I think that speaks to the idea that you need that degree of flexibility. It’s fine to have a goal in mind, but the moment I start blocking out particular bits of time, the whole thing doesn’t survive contact with reality and people get very disheartened.

You suggest the real scandal in the Volkswagen emissions cheating is not that VW found a way to cheat a predictable test. What was it?

Well, that is scandalous, but it’s not surprising. It’s been done before. In the late 1990s, most of the people who make trucks in the United States cheated emissions tests. This has all happened before, and yet the test is still incredibly predictable. So it’s also scandalous that the tests are so easy to predict. We would never accept an exam where precise details were released to students in advance. All you’re doing is testing someone’s ability to memorize an answer. That’s not OK. Yet somehow it’s fine to do the same thing when we’re testing emissions.

We have a very close parallel in the financial system. The Federal Reserve imposes tests on the banks — the stress tests. These are scenarios: interest rates go up 2 percent, the dollar depreciates 10 percent, housing prices fall 20 percent, here’s a list of bad stuff that happens. How does your balance sheet look now? 

The book is not just about creativity. It’s also about how these organizational systems can make us very fragile, very vulnerable. We rehearse for a particular circumstance, and then when a slightly different circumstance shows up, we’re in trouble. The reason this tends to occur is because both the regulators and the regulated likes things that are predictable. The predictability protects the regulators as well. They can say “look, everything is very transparent. This is what we did. We asked all the right questions." The moment the regulators start throwing in curveballs, they probably become better regulators. But it’s much harder to defend what they’re doing.

You write about Donald Trump in your book. Has your view changed on his campaign in recent weeks?

He clearly hasn’t had a great campaign in the last month. It’s been much more difficult for him to keep control of the news cycle against Hillary Clinton. Unpredictability itself starts to become predictable. He’s run out of road, and many of the people I describe in the book using this tactic eventually run out of road.

But I think it's easy to forget how far he’s come using unpredictability as a tactic. When he began the campaign, it was a joke. He was given no chance — [everyone thought] the nomination would clearly go to someone like Jeb Bush. Yet somehow he managed to break through. He was willing to keep improvising and take risks. He was fascinating to the media. He kept tripping up Marco Rubio and Bush: He would say something, and they would say “Gosh, we’ve got to go talk to a focus group,” and then he would say something else and they just couldn’t keep up with him. It’s a very effective strategy. I don’t endorse it. I’m not saying it makes the world a better place. But I am saying it can work, using disorder as a weapon.

What do you think is the biggest lesson for leaders in the book?

I would encourage them to think about Martin Luther King Jr.’s experience. MLK really broke through as a communicator when he let go of his scripts. He was a good speaker when he was writing and memorizing his scripts. But when he stopped and began to improvise, that’s when he really started speaking with tremendous force. Each speech felt more real. He could respond to the energy he was getting from the room. The second half of the ‘I have a dream’ speech was improvised. I wasn’t aware of that when I started writing the book, and that really stuck with me.

Read also:

Like On Leadership? Follow us on Facebook and Twitter, and subscribe to our podcast on iTunes.