Finding Innovation Through Failure
Technology products are only effective if they work—not occasionally, but all the time, and in any possible scenario. This is particularly true for next-generation vehicles. The vehicles are characterized by a series of digital safety features, including automatic emergency breaking, blind spot warning, pedestrian detection, electronic stability control, lane departure avoidance and more. Known collectively as advanced driver assistance systems (ADAS), the innovations are meant to mitigate the potential of human error in driving, and thus reduce accidents. For technologies that function to literally save lives, categorical efficacy is imperative.
So before next-generation vehicles make it on the road, the technologies they’re equipped with go through rigorous testing. Dr. Furea Shirai is one of the people responsible for designing and executing those tests. Shirai works at NI, a technology company that develops automated test and measurement systems and virtual instrumentation software for a range of industries, from transportation to wireless infrastructure to aerospace. As a senior program manager in the company’s ADAS sector, Shirai partners with car manufactures to ensure automated safety features will keep people protected. She and her team help these firms test everything from software interoperability to whether a radar can recognize the difference between a human and a plastic bag. “We want to make sure that the systems respond the way they’re supposed to,” she said.
To do so, they take what might be considered an unexpected approach: The engineers actually seek out problems in the tech. They look for every possible way a system may break, miscalculate or misfire, often even introducing variables meant to trigger malfunctions. To take one example, Shirai’s team intentionally stalled a fan, just to see how the car would respond. And it’s not just a feature of automotive manufacturing, but rather all engineering sectors. This method—finding failure in one’s own products—may seem counterintuitive, but it’s actually an essential step in the pursuit of groundbreaking innovations.
Dr. Taylor L. Riché
NI Software Section Manager
“One quote that has been attributed to Thomas Edison is, ‘I have not failed. I've just found 10,000 things that didn’t work.’ And I think it’s important in the mind of an engineer to understand that, really, it’s about tolerance for failure to get to an end goal,” said Charles Schroeder, a business and technology fellow at NI. “The ultimate goal is not to find failure. The goal is to create an amazing breakthrough product. But if you’re not willing to have failure or to root it out, you’ll never get there.”
The essentials of the engineering process
According to Schroeder, the engineering process typically starts with an idea, often arising from a problem or an unmet customer need. Engineers then brainstorm different ways to address it. “Once you get an idea, then you need to frame it both technically and commercially,” he said. “What are the things we know and don’t know about the technology to find a solution.”
This ideation then leads engineers to creating a prototype—something of a first draft of a product or service. It’s at this stage that the testing begins.
Dr. Taylor L. Riché
NI Software Section Manager
“Test is a key part to engineering and to solving these problems because, at the end of the day, you have to show that your solution actually works,” said Dr. Taylor Riché, a software engineer and section manager at NI. “If you don't have a robust test program to go along with the engineering of your product, you're really going to fail.”
Test across a product lifecycle is not a simple process—it’s not meant to be. Engineers have to try to forecast any possible point of failure. Imagine trying to design the technology that supports 5G wireless infrastructure and having to protect against every possible disruption to the network. It’s common for engineers to have reconceptualize and redesign prototypes countless times before a product is ready for market. “It seems like getting the first idea to work might be the most challenging part, but no, it’s that nasty backend of refining and refining and refining so that you can build a hundred million copies with really high quality,” Schroeder said.
It’s this tricky yet essential stage in the innovation process that NI software is designed to support.
Cheryl Tulkoff
NI Director of Corporate Quality
How software supports innovation
Today, advanced software plays a fundamental part in supporting the product lifecycle process. For example, in order to test ADAS features in a safety vehicle, automotive companies need a way to record and monitor the sensors on a test drive. That’s where NI comes in, explained Shirai. The company’s software aids in documenting performance information, which is then used to make improvements to the technology in the vehicle. “The software is integral to gathering the data and evaluating it,” said Cheryl Tulkoff, director of corporate quality and continuous improvement at NI. “In the failure analysis process, it's how you determine what didn't behave right.”
Cheryl Tulkoff
NI Director of Corporate Quality
Meanwhile, sometimes software does the assessments on its own, without a prototype. A testing process called digital engineering, or model-based engineering, is becoming more prevalent, explained Riché, in which innovators create a model for how a physical device will perform using only software—allowing innovators to test products more efficiently and cost-effectively. Riché cites a plane as an example. “If a plane model that’s being tested fails in the air, even if no one’s on it, that is millions and millions of dollars that are gone. But if it fails as a [software] model in a test, you don’t lose any equipment. You’ve found that failure early,” he said.
It’s those moments of failure—enabled by analytics-based testing throughout the product lifecycle—that lay the foundation for the true objective: safe and effective world-changing technologies.
“In innovation, if you never fail, you're probably not trying hard enough,” said Tulkoff. “It’s a mindset that needs to be integrated very early, ‘how can I make this fail?’ Because looking for weak spots or places where things are vulnerable is only way to truly develop technologies that are built to last.”
Dr. Taylor L. Riché
NI Software Section ManagerFailure is incredibly important to the testing process because the earlier you fail a test, the sooner you fix that problem, and this is why early test and thus early failure saves money. It saves lives. It saves equipment. Because we can get failures in early tests that are just say modeling tests, or early tests where we're just using a component and not the entire system. Again, the earlier we find those failures in our tests, the better overall product we're going to have, and the better overall solution we're going to have, and the cheaper that solution is going to be to build.
Dr. Taylor L. Riché
NI Software Section ManagerI think software data and analytics is becoming more and more involved in testing and engineering because we can get a lot more data now than we used to be able to, and we can process it more effectively. And so now you can actually monitor and end up with a lot more data earlier in your process and model that, and actually detect failures much sooner than you could before. Where you used to depend on a physical item actually failing, now you have enough data that you can analyze with different modeling techniques that you can actually understand your failures much earlier, saving money, saving costs, saving lives even. And that data analysis just wasn't possible before, but now with both the techniques and this hardware that we have, we can actually do a lot better engineering than we used to be able to do.
Cheryl Tulkoff
NI Director of Corporate QualityThere are a lot of misconceptions around finding failure in the test engineering process... When we talk about it from the development and the innovation perspective, the goal is to find failure, is to stretch the testing, the boundaries that you do. So you look for opportunities to make a product more robust, make it more stable. That's one of the things that I try to preach in my role is that if you're not finding a failure, you're probably not trying hard enough. And if you designed a product right the first time, you're probably not stretching the boundaries of an innovation to really create something new, dramatically different requires that you look for failure. And I think the people who are really innovators understand that and embrace that concept.
Cheryl Tulkoff
NI Director of Corporate QualityPhysical testing and virtual testing are both key components to the test engineering process. Physical testing has been around with us since the beginning of time. You have an actual prototype in various stages that you can subject to forms of software testing, but what we have today is the unique opportunity to now do virtual testing. Things that allow us to test earlier, faster and more cheaply than ever before, because we don't even need even a small component of a prototype physically to be working. We can do it all in software, in a model, inject faults, take a product or a process through its phases, refine and revise it. So when we do get to physical prototyping, we can narrow in on potential areas of weakness or areas where we'd like to improve that robustness and validation in the physical world.