“I loved the creativity of Lego,” said Shifrin, now 22, whose parents guided him through builds for most of his childhood in Massachusetts. “Plus, it was wonderful brain training for me: Blind people have trouble with spatial reason and spatial awareness, and Lego lets you go piece by piece to put a room together.”
The Lego-stuffed crate did something else for Shifrin, too: It set him on a mission to make Lego play accessible to vision-impaired children around the globe. His years-long dedication paid off Wednesday when the Lego company debuted its first audio and Braille building instructions, a project completed with Shifrin’s input — and because of his persistence.
The instructions, available free online, come in English and detail how to put together four different toy sets: Lego Classic, Lego City, Lego Friends and Lego Movie 2. Customers simply go to Lego’s new website and choose either audio or Braille. The Braille guidelines can then be translated by a “Braille reader,” a device that converts on-screen computer text to raised Braille characters.
“This was always something we wanted to address: How do we make Lego for visually impaired and blind children?” said Fenella Blaize Charity, Lego’s creative director. “We want the Lego experience out to as many kids as possible.”
Lego will test the instructions over the next several months and gather feedback from vision-impaired users. At the end of the year, the company will reassess, fix any bugs and — if all goes well — eventually develop audio and Braille instructions for every Lego set it sells, Charity said. It will also translate the instructions into more languages.
The project — at a cost Lego declined to disclose — is being funded by the Lego Foundation, a division of the Denmark-based company that focuses on empowering and educating children through play. It’s the culmination of months of work by Lego employees, who partnered with staffers at the Austrian Research Institute for Artificial Intelligence and Shifrin himself. Lego began exploring the idea after Shifrin reached out roughly two years ago.
The challenge the team faced was “pretty massive,” Charity said. Lego is an “extremely visual” form of play, she said, given that the entire experience centers on seeing what you’re building.
Even with a helping parent or friend, it can be difficult for a blind child to locate and fit bricks together in the right order. The number of pieces in just one Lego set can total in the thousands. To top it off, Lego’s instruction booklets do not use text: They rely solely on pictures.
“We really just felt, ‘How do you even start with this project?’ ” Charity said. “It was only when Matthew came to us with this amazing solution he’d already started work on that we could feel it was possible.”
Shifrin and Finkel, his sitter, came up with their “solution” on Shifrin’s 13th birthday. Finkel had given Shifrin the “Battle of Alamut,” a Middle Eastern-style Lego palace and a typical offering for the teen, who had remained a Lego fanatic (and still is). But she paired the gift with what proved a priceless treasure.
Finkel invented a unique name for every one of the more than 800 pieces in the palace set. Then she spent countless hours compiling a binder’s worth of building instructions in Braille that spelled out how to fit them all together.
For the first time in his life, Shifrin sat down at his kitchen table, binder close at hand, and completed a Lego build without any help, start to finish. He was so engrossed he canceled a scheduled math lesson to keep building — it was a “morning-to-night kind of session,” he said.
“I never thought I would be able to build Lego sets independently,” Shifrin said. “If you’re told all your life or know deep down that you couldn’t do something — and then suddenly there’s a way for you to do it …” He trailed off.
“It’s just an incredible feeling,” he said.
Shifrin knew immediately that he had to help other blind children find the same independence. Over the next half-decade, he and Finkel created similar instructions for about 45 other Lego sets, all published on a website they spun up, “Lego for the Blind.”
The duo eventually shifted away from Braille, establishing a system where Finkel typed up instructions and Shifrin tested them out by listening and following along using his screen reader. They worked together at every spare moment — on weekends and during Shifrin’s school vacations — and each set of guidelines took them at least a month to complete.
Shifrin began reaching out to Lego several years ago to discuss his work, and in 2017 he finally connected with someone in the Creative Play Lab — a division of the company that invents new toys — setting everything into motion. The program debuted by the company this week is a high-tech version of what Shifrin and Finkel did for years on their own as “just two people in a living room,” as Shifrin put it.
The new system is far less time-consuming and labor-intensive. Working with Lego, employees at the Austrian Research Institute figured out a way to take the code that anchors the company’s instruction booklets and transform it into audio and Braille building guidelines. Shifrin offered feedback, for which Lego paid him.
To make its picture-based instructions, Lego generates 3-D representations of each of its sets, explained Friedrich Neubarth, a researcher at the institute. Each of those figures contains helpful data on the pieces that make up the set and how those should fit together.
“So we reverse-engineered that data and created a program that tries to understand how these models are set up and generates text that not only gives instructions, but gives information about structure,” Neubarth said.
Lego and the institute are working to perfect the program, which still has a few flaws. Right now, for example, it requires human input at certain points — a feature Neubarth hopes to eliminate in the future. If he succeeds, Lego will be able to generate instructions for all of its toy sets with little effort.
“That’s the goal: We want to make Lego something that every kid can tap into,” Charity said.
Shifrin is thrilled to see his vision of a more inclusive Lego become a reality. He just wishes Finkel — who died after a battle with cancer about two years ago — were here to see it.
Lego kept the two glued together in Finkel’s final months. Even as her health deteriorated, Finkel was determined to keep working and finalize more instructions. Shifrin often visited her house, even when she was in hospice care, and built sets by her bedside in an effort to take Finkel’s mind off her cancer.
When she died, Finkel left half-written instructions for one final Lego project on her laptop.
It was Finkel’s death that spurred Shifrin to try contacting Lego one more time — at last, with success. Shifrin said he processed his grief for his “best friend” by diving into the new project with Lego.
“I think she’d be very glad that we came this far,” Shifrin said. “We’d always hoped that it would — we weren’t sure it would — but I think she’d be happy.”