Somehow, we can grow pessimistic about life in general while continuing to be upbeat about our own future. (TWP)

We all like to think of ourselves as rational creatures who smartly prepare for the worst. We watch our back, weigh the odds and pack an umbrella when the skies look threatening. But although we take such precautions, we generally expect things to turn out pretty well — often better than they actually do.

The belief that the future will probably be much better than the past and present is known as the optimism bias, and most of us have this tendency to overestimate the likelihood of good events happening to us and underestimate the likelihood that bad events will come crashing down.

For instance, people hugely underestimate their chances of losing their job or being diagnosed with cancer. They also envision themselves achieving more than their peers and overestimate their likely life span, sometimes by 20 years or more.

In short, we are often more optimistic than realistic.

Take marriage, for example. In the Western world, divorce rates are higher than 40 percent: Two out of five marriages end in divorce. But newlyweds estimate their own likelihood of divorce at zero. Even divorce lawyers, who should know better, hugely underestimate their own likelihood of divorce. Although the sunniest optimists are just as likely to divorce as the next person, they are also more likely to remarry. In the words of the 18th-century English author Samuel Johnson, “Remarriage is the triumph of hope over experience.”

Many of us who have children believe that our kids will be especially talented, even while thinking our neighbor’s kids aren’t all that promising. A survey conducted in 2007 on behalf of the BBC found that 93 percent of respondents were optimistic about the future of their own family, while only 17 percent were optimistic about the future of other families. Collectively, we can grow pessimistic — about the future of our fellow citizens, about the direction of our country, about the ability of our leaders to improve education and reduce crime — while we continue to think our own future is bright.

Why does optimism about our personal future remain incredibly resilient? It is not that we think things will magically turn out okay for us, but rather that we believe we have the unique abilities to make it so.

The rosy future

Optimism starts with what may be the most extraordinary of human talents: mental time travel, the ability to move back and forth through time and space in one’s mind. To think positively about our prospects, it helps to be able to imagine ourselves in the future. Our capacity to envision a different time and place is critical for our survival. It allows us to plan ahead, to save food and resources for times of scarcity, and to endure hard work in anticipation of a future reward.

While mental time travel has clear survival advantages, conscious foresight came to humans at an enormous price — the understanding that death awaits. The knowledge that old age, sickness, decline of mental power and oblivion are somewhere around the corner can be devastating.

Ajit Varki, a biologist at the University of California at San Diego, argues that the awareness of mortality on its own would have led evolution to a dead end. The despair would have interfered with our daily function, bringing the daily activities needed for survival to a stop. The only way that conscious mental time travel could have arisen is if it emerged along with irrational optimism. The knowledge of death had to emerge in parallel with the persistent ability to picture a bright future.

The capacity to envision that future relies partially on the hippocampus, a brain structure that is crucial to memory. People with damage to the hippocampus are unable to recollect the past; they are also unable to construct detailed images of future scenarios. The rest of us constantly voyage back and forth in time; we might be thinking of a conversation we had with our spouse yesterday and then immediately jump to our dinner plans for later tonight.

But the brain doesn’t travel in time randomly. It tends to engage in specific types of thoughts: We consider how well our kids will do in life, how we will obtain that desired job, whether our team will win, and we look forward to an enjoyable night on the town. We also worry about losing loved ones, failing at our job or dying in a plane crash. But research shows that most of us spend less time mulling over negative outcomes than we do over positive ones. When we do contemplate defeat and heartache, we tend to focus on how these can be avoided.

Why do we maintain this rosy bias even when information challenging our upbeat forecasts is so readily available? We experience both positive and negative events in our lives. We know the economy is unstable, for example, but still we remain optimistic about our own future. When expectations are not met, we alter them. This should eventually lead to sober realism, not blind optimism.

Underestimating bad news

Only recently have we been able to decipher this mystery. My colleagues and I at University College London recently scanned the brains of people as they processed both positive and negative information about the future.

Among other things, we asked them to estimate how likely they were to encounter 80 different negative events in their life, including developing cancer, having Alzheimer’s disease and being robbed.

We then told them the likelihood that a person like them would suffer these misfortunes; for example, the lifetime risk of cancer is about 30 percent. Then we asked again: How likely are you to suffer from cancer? We wanted to know if people would change their beliefs according to the information we provided. It turns out they did, but mostly when the information we gave them was better than they had expected.

If someone had estimated that their risk of cancer was 50 percent and we told them, “Good news: The average likelihood is much better, only 30 percent,” the next time around they would say, “You know what? Maybe my likelihood is only 35 percent.” So they learned easily and quickly.

However, if someone started off estimating their cancer risk was 10 percent and we told them, “Bad news: The average likelihood is about 30 percent,” they would scale up only gradually. The next time, they might say that their likelihood of contracting cancer was only 11 percent. It is not that they did not learn at all. They simply decided that the figures we provided were not pertinent to them.

Where do these irrational beliefs come from? This disconnect is related to something scientists call prediction errors, which describe the difference between what you expect and what actually happens.

When we gave our research volunteers information about future likelihoods (such as contracting cancer), we scanned their brains looking for changes that might relate to the gap between their estimates and the information they received.

A few brain areas, including the left inferior frontal gyrus 1, responded to unexpected good news. For example, when someone thought his likelihood of cancer was 50 percent and we told him it was only 30 percent, this region responded fiercely.

On the other side of the brain, the right inferior frontal gyrus responded to unexpected bad news. But it did not do a very good job. In fact, the more optimistic a person was, the less this region seemed to process bad news. If your brain is failing to respond to unexpected bad news, you are constantly wearing rose-tinted glasses.

These findings are striking: When people learn, their neurons encode desirable information that can enhance optimism, but the neurons fail at incorporating unexpectedly undesirable information. When we hear a success story such as that of Facebook founder Mark Zuckerberg, our brains take note of the possibility that we, too, may become immensely successful and rich one day. But hearing that the odds of divorce are almost one in two tends not to make us think that our own marriage may be destined to fail.

Does age matter?

Does everyone show an optimistic bias? As it turns out, they do. In an as yet unreleased study, my colleagues and I found that people of all age groups changed their beliefs more in response to good news, and they discounted bad news.

Even more surprising was the finding that kids and elderly people both showed more of a bias than college students. On one hand, the young and the old were quite good at responding to desirable information: Everyone updated their beliefs similarly when they learned they were less likely to get cancer or have their credit card stolen than they had initially believed. But when they learned their chances were worse than expected, kids, teenagers and older adults seemed to ignore this information more than college students and middle-aged individuals.

The behavioral economist Andrew Oswald has found that from about the time we are teenagers, our sense of happiness starts to decline, hitting rock bottom in our mid-40s. (Middle-age crisis, anyone?) Then our sense of happiness miraculously starts to go up again rapidly as we grow older. This finding contradicts the common assumption that people in their 60s, 70s and 80s are less happy and satisfied than people in their 30s and 40s.

How can we explain this? The first thing that comes to mind is that these changes have something to do with raising kids in our 30s and 40s. Could it be that having children in the household has a negative influence on our happiness?

Oswald ruled out this possibility. He also controlled for people being born in better times, marital status, education, employment status, income: The age pattern persisted. Even more surprising, the pattern held strong even though Oswald did not control for physical health. In other words, older individuals are happier and more satisfied than middle-aged individuals even though the health of the former is generally worse.

Oswald tested half a million people in 72 countries, in both developing and developed nations. He observed the same pattern across all parts of the globe and across sexes. Happiness diminishes as we transition from childhood to adulthood and then starts rising as we grow wrinkles and acquire gray hair.

And it’s not only we humans who slump in the middle and feel sunnier toward the end. Just recently, Oswald and colleagues demonstrated that even chimpanzees and orangutans appear to experience a similar pattern of midlife malaise.

Oswald did observe some interesting differences. For one, the age at which happiness is at its lowest is different around the world. In Britain, for example, happiness reaches rock bottom quite early — at 35.8 years of age — before it starts going up again. In Italy, by contrast, happiness hits its ultimate low much later — at 64.2 years. And while women reach the bottom of the happiness barrel at 38.6 years on average, men reach it more than a decade later — at 52.9 years.

(Oswald observed another interesting divergence in happiness trends: Americans have been growing less happy since 1900. In Europe, however, happiness has been increasing steadily since 1950, after 50 years of decline. Why the difference? We simply don’t know.)

What explains the age findings? One possible answer is that happy people live longer, while pessimistic ones die earlier, so those elderly individuals who remain for scientists to test are happier than the average 30- or 40-year-old.

Another possibility is that older individuals have experienced a larger range of adverse events, so they are less likely to view these events as frightening and consequential; thus, their psychological coping mechanisms may be better.

A third potential explanation is that the decreased ability in older adults to take bad news into account may be enhancing their optimism and thus increasing their happiness. The decline may be connected with age-related changes in frontal lobe function, which is important for incorporating new information into prior beliefs.

The sun will shine

Why would our brains be wired in a way that makes us prone to optimistic illusions? It is tempting to speculate that optimism was selected by evolution precisely because, on balance, positive expectations enhance the odds of survival.

Research findings that optimists live longer and are healthier, along with the fact that most humans display optimistic biases — and emerging data that optimism is linked to specific genes — all strongly support this hypothesis.

But the optimism bias also protects and inspires us: It keeps us moving forward, rather than to the nearest high-rise ledge. To make progress, we need to be able to imagine alternative realities, and not just any old reality but a better one; and we need to believe that we can achieve it. Such faith helps motivate us to pursue our goals.

Sharot is a research fellow in cognitive, perceptual and brain sciences at University College London and author of “The Optimism Bias: A Tour of the Irrationally Positive Brain.” This article was excerpted from the new TED e-book “The Science of Optimism.”