By now, the VCR story is familiar. It's an unsettling tale of an innovation gap between U.S. and Japanese companies. Americans developed the basic technology of the VCR, but Japanese companies commercialized it. They now dominate an immense market: In 1987, Americans bought an estimated 12 million VCRs.

The videocassette recorder was no freak accident. Anyone who thinks it was will be disappointed by a new study from economist Edwin Mansfield of the University of Pennsylvania. Innovation isn't inventing; it's converting technology and new ideas into viable products. Mansfield finds that Japanese companies do this faster and less expensively -- at least 10 to 20 percent less expensively -- than similar U.S. companies.

Why? Americans haven't become unimaginative. Our basic research -- the quest for knowledge for its own sake -- is still acknowledged to be the world's best. The climate for new entrepreneurial companies is inviting; indeed, these companies generate many new products. Nor is low spending on research and development to blame; our R&D spending exceeds the combined total of Japan and West Germany. The problem lies mainly with large companies, which do most of the R&D.

For many, innovation creates a Catch-22. Engineers and scientists prefer to work on major breakthroughs, which are exciting and challenging. But corporate executives are leery of the huge investments and risks associated with entirely new projects. A standoff results. Improving existing technologies and products suffers from low status. But big new projects get bogged down in corporate politics and bureaucratic planning.

The VCR story illustrates what goes wrong. The U.S. firm that invented the videotape recorder, Ampex, specialized in expensive machines for broadcasters. It had little interest in developing a product for a mass consumer market. Meanwhile, RCA and CBS attempted to perfect entirely new technologies that would allow viewers to play prerecorded programs on their televisions.

By contrast, the Japanese tinkered with the basic Ampex technology. In 1965, Sony introduced a videotape machine for consumers. Other companies followed. Many of these early machines were flops. But from them, the Japanese learned what features were necessary for success. "The American companies had the projects in the industrial labs. They spent years and years and tons of money without ever putting anything on the market," says James Lardner, author of "Fast Forward," a history of the VCR.

As Lardner points out, the Japanese didn't simply copy the U.S. technology. Many of their changes involved crucial improvements that made VCRs smaller, more reliable and less expensive. When RCA finally marketed its VideoDisc technology in 1981, it was too little, too late. The company ultimately abandoned the product at a total loss of more than $500 million.

Mansfield's study suggests that the VCR episode isn't unique. The study covered 30 firms in each country, and not all the news is bad for Americans. In some industries, notably chemicals, there are few differences between American and Japanese companies. And companies in both countries do equally well at introducing products based primarily on their own research.

The great Japanese strength lies in developing products from existing technologies. Like the VCR, these products often aren't copies; they involve major refinements. Japanese costs are about 50 percent lower and introduction times 30 percent shorter. Much of the Japanese advantage stems from less spending on marketing studies designed to discover what consumers want.

One obvious need is for U.S. companies to pay more attention to foreign technology. When U.S. industries enjoyed global leadership, they ignored developments abroad. To do that now is suicidal; there are too many good ideas elsewhere. Yet, bad habits linger. In 1983, only 10 percent of large U.S. machinery manufacturers spent as much monitoring international technology as did the average Japanese machinery firm.

Watching foreign markets for new ideas usually requires being there, either through exports or local production. It's a mistake to think that only big companies can manage this. A study by the American Business Conference -- a group of medium-sized firms -- found that many member companies had gone overseas in the first years of their businesses. Innovation also transcends high-technology. Dunkin' Donuts introduced small kiosk-type stores in the United States only after discovering them abroad.

The greatest need, though, is for U.S. companies to become less compartmentalized. Products succeed when there's a sharing of information and enthusiasm across the boundaries of corporate fiefdoms. RCA's VideoDisc failed in part because the company's industrial laboratory, where the machine was developed, and the rest of the company were suspicious of each other.

"In too many corporations any business opportunity that originates in the laboratory is automatically suspect. Researchers ... are believed to be incapable of sound commercial judgment," writes Margaret Graham of Boston University in "RCA & the VideoDisc." "Often the negative stereotypes cut both ways. Until recently, the engineer who 'dirtied his hands' working in a plant could not possibly be a high-class engineer."

At the Japanese electronics companies, there is more cooperation and informality. American companies often use marketing studies for political purposes: to settle disputes over which products should be developed. The Japanese recognize that these studies can be time consuming, expensive make-work. The best way to find out whether a product will succeed is to try to sell it. People don't know what they want until it exists.

The innovation gap mostly reflects bad management. American companies ache for great innovations, but recoil at the risks. Marketing studies are supposed to resolve the contradiction by predicting the unpredictable. In practice, they're a formula for spending more on innovation and getting less.