It’s amazing how quickly social norms have (temporarily) changed. Just a few weeks ago, people wearing masks were few, far between and subject to multiple dirty looks and whispered asides (Why didn’t you donate that? Don’t you know it doesn’t really work?). Now, in many states, wearing masks is the law. And millions of people are listening. The home mask-making industry is one of the few vibrant corners of the economy right now, as is — unexpectedly — the bandanna sphere. If you step outside right now (and please only do so if you need to!) you will see a sea of masks. We are all doing it.

But we don’t really like it.

The mask has been portrayed by protesters as a symbol of government oppression, rather than lifesaving gear. But there are also more common complaints: It’s harder to breathe with a mask on. Our glasses fog up. Our noses get itchy. Our ears get pulled. Our voices are muffled.

But, most significantly, we can’t smile at each other, use any facial expressions or even see each other’s faces, giving even brief, socially distanced social encounters a cold, eerie cast.

Not being able to see other people’s faces challenges a crucial part of how we communicate. Rooted in the ancient practice of physiognomy, which links external facial features to internal character, people have long built relationships and assessed others based on how they look. The long history of physiognomical practice — which saw its height in the 19th century but lingers through today — demonstrates the tremendous stock we place in faces as an index to character, and helps explain why we are so uncomfortable in masks today, even if they are a temporary necessity.

Dating from the 16th century, the term “barefaced” described someone who was beardless or maskless and thus open, unconcealed and honest. Telling a “barefaced lie” through the 19th century was a particularly egregious offense, being as it was so shameless and unconcealed, paving the way for today’s equally impudent “boldfaced lie.”

In 1802, the novelist Maria Edgeworth wrote about those who “call a good countenance the best letter of recommendation,” speaking to the mistrust of those who covered, concealed or in other ways manipulated their faces.

Charles Darwin’s face was among the most famously read for physiognomical clues: He was almost prevented from joining the journey on the HMS Beagle, the trip that gave him the material to develop his theory of evolution. The captain of the ship, Robert FitzRoy, was an avid physiognomist and felt that Darwin’s nose was too short, reflecting a lack of determination to see the arduous journey through to completion.

Reading facial features to judge a person’s character reached its height in the 19th century, with the unprecedented urbanization and industrialization that produced some of the biggest cities in the history of the world. People in the madding crowd needed a way to make judgments about others fast.

Physiognomy, in consonance with other reductionist evaluative practices like craniometry (skull measurements) and phrenology (brain bumps) provided a seemingly empirical way to classify both individuals and groups. In reality, these supposedly empirical approaches reflected underlying biases, with fundamental assumptions about race and class framing both the approach and the findings. A brief example: The physiognomical categories of “Roman nose” and “Jewish nose” were both aquiline and beaklike, yet one was indexed to nobility and the other to avarice.

By the end of the 19th century, Western classification practices shifted from individual assessments to large group categorizations with the rise of statistics, census practices, eugenics and social Darwinism. Individual physiognomical readings declined. It became less common to openly assess an individual’s character simply by looking at her face. Yet remnants of the practice remained, with phrases like “beady-eyed criminals” and notions of noble-jawed heroes continuing to shape people’s assessments of character through physical features.

In many ways, the idea that we can measure and evaluate people’s physical features to determine something about who they are remains with us today, often in deeply problematic, reductionist, racist, sexist and homophobic ways. Attempts to assess, for example, sexuality based on finger length, or levels of aggression based on face width are recent examples of modern physiognomy. And more abstract notions of what kinds of faces and skin colors are desirable permeate our approach to appearance. These assessments are not about expression or communication, but actual static features and the assumptions we build into which ones we think are better.

We look at people to know them. When we can’t look at them, we feel we know and trust them less. And when we can’t show our own faces to the world, we may feel we are hiding something.

So what do we do at a time when we need to wear masks in public but we also need to connect with one another while maintaining social distancing?

A group of doctors at Israel’s only dedicated coronavirus hospital have an innovative solution to this problem. They have started wearing images of their faces on their protective gowns. While these faces serve no purpose in terms of communicating expression or reactions, they appear to make patients feel better. Knowing what their doctors look like provides patients a sense of comfort and familiarity, even as they are static representations.

Designer and artist Danielle Baskin has a different response to this need, offering personalized masks that have a picture of the obscured part of the wearer’s face on the outside. Eventually these could be used to confuse face recognition technology or experiment with nonsurgical ways to alter appearance, but for now, they help people feel like they aren’t hiding their faces. Which helps other people feel like they know those they are looking at.

“Faced” masks don’t help with the nose tickles, voice clarity or foggy glasses. But they may make wearing masks less uncomfortable, especially when we actually get to see other people — or part of them — on a regular basis again. For now, we should absolutely trust people wearing masks more than those whose faces we can see. Masks will continue to itch, but the greatest unease is not physical; it’s historical.