|
|
|
|
||
|
Data Mining Digs In
By Claude J. Bauer
A recent study by the Stamford, Conn.-based market research firm META Group predicts that the data mining market will grow from $3.3 billion in 1996 to $8.4 billion by the year 2000. According to industry experts, data mining's 150 percent growth rate will create opportunities for systems integrators, consultants, and other IT professionals, as well as software vendors that design and sell data mining tools. Data mining's explosive popularity is based in part on the maturing discipline of data warehousing. "Companies have invested heavily in the last two years in data warehouses," said David Harris, business manager for data warehousing at the Burlington, Mass. offices of Sybase Inc. Now, many of those companies are looking to data mining as a way to gain strategic value from warehoused information. A new crop of inexpensive, easy to use tools is also driving the data mining market. "Data mining is a very hot area," said Christopher Eastman, practice manager for the Bethesda, Md.-based consultancy Virtulogic Inc. "Big companies want to [conduct data mining], but don't know where to get started," he said. According to the META Group study, fifty percent of global 2000 enterprises believe data mining will be "critical to their goals" over the next two years. "Within five years, [data mining] will be as important to running a business as the business systems are today," predicted Evangelos Simoudis, worldwide vice president of solutions for IBM's global business intelligence division. IBM analysts concur, and estimate that by 2000, companies worldwide will invest $70 billion in business intelligence technologies, which include data mining. Simoudis believes data warehousing and data mining tools are experiencing "tremendous growth" because companies are now beginning to capitalize on their warehoused data. Why Data Mining? The business intelligence that results from data mining has a variety of uses, ranging from fraud detection and customer churn analysis to credit analysis. "The payback for a company is tremendous," Simoudis said. For example, he notes that 10 percent of the $1 trillion spent annually in the U.S. on health care represents fraudulent claims. If data mining initiatives helped insurance companies isolate only one percent of that amount, it would represent a significant amount in savings, he said. According to Ram Srinivasan, vice president of product marketing for the San Mateo, Calif.-based vendor DataMind Corp., data mining also can pay big dividends when used to analyze customer retention. For example, commercial long distance telephone companies now suffer a 30 to 40 percent subscriber churn rate, he said. "Preventing this could save millions of dollars," Srinivasan noted, since the cost of recapturing a single long distance customer can range as high as $400. "It all boils down to getting value out of the data," said Mark Brown, program manager for data mining at SAS Institute Inc. Speaking from the company's Cary, North Carolina headquarters, Brown stresses that data mining is fast becoming a "mission-critical" activity for many corporations. What is Data Mining? According to Srinivasan, data mining, "turns data into information, information into knowledge, and knowledge into action." Data mining goes hand-in-hand with data warehousing, which enables users to collect and store data, as well as prepare it for analysis. Some data warehouses now reach into hundreds of terabytes (trillions of bytes) of data. "It's no longer a question of running out of data, but drowning in the data you've created," Srinivasan said. However, thanks to a bumper crop of user friendly data mining tools, companies can now approach data mining at different levels. Some firms may opt for expensive macromining, which involves in-depth mining to produce a few high-yield nuggets of information, while others may choose less costly desktop data mining, or micromining, where business technologists use PC-based products to extract slices of data for marketing efforts. Some vendors, such as DataMind, are aiming at the market between these two, where moderately priced data mining tools enable companies to comb through large databases and cull strategic information without having to rely on outside help. Since data mining efforts typically revolve around stimulating business, the types of data most often mined includes transaction data, demographics, and responses to previous promotions. Also, data mining is "largely concerned with predictive models," said Herb Edelstein, president of the Potomac, Md.-based consulting firm Two Crows Corp. For example, data mining might reveal which credit card customers are most likely to add new services to their accounts, or which business travelers would be most likely to vacation in Europe. After companies create data warehouses and mine the data, decision support tools come into play. These take the patterns and trends discovered through data mining and help users apply those findings to corporate decision-making. "The biggest challenge about data mining is that you have to be willing to make strategic changes," in the company's business practices, Harris said. Too Many Tools John Ladly, senior program director for META Group, points out that the data mining market is now crowded with dozens of products, many of which are new or recent entries into the market. For example, IBM's flagship data mining toolkit Intelligent Miner was introduced as recently as 1996. Data mining vendors now include large companies such as IBM and Digital Equipment Corp., as well as a host of smaller firms such as Information Discovery Inc. and DataMind. With upwards of 70 vendors selling data mining products, "There's got to be a big shakeout," said Andrew Braunber, editor of the Bethesda, Md.-based trade publication Data Mining Magazine. SAS Institute, the most recent entry in the market, is preparing a Spring 1998 launch for Enterprise Miner, a software package now in beta testing at 25 companies nationwide. According to Brown, Enterprise Miner will enable users to examine customer retention and acquisition trends, evaluate cross-selling practices, add services to existing accounts, and build customer loyalty programs. Data Mining Skills Not unlike other technical disciplines, there is, "definitely a shortage of people who can work in [the data mining] environment," Simoudis said. He counsels that those who want to break into data mining have strong quantitative skills that go beyond a cursory familiarity with statistics. To produce usable results, data miners must often draw on advanced analytical approaches such as predictive modeling, supervised induction, association discovery, sequence discovery, and conceptual clustering. Edelstein believes, "The key to success in data mining is really understanding the data." He adds that it is also important to know how to state the problem correctly before mining begins, since the goal of data mining is to extract meaningful patterns and trends from the data, not just find random correlations. Edelstein suggests that those who want to break into data mining from the IT side should learn how to build data warehouses and prepare data for mining. Brown agrees, and adds that in data mining, there is a strong demand for workers with business knowledge plus analytical skills. He suggests that IT professionals who want to add data mining skills to their tool kit "really understand warehousing concepts," and "reach out to business users" for information about their company and the data it needs. He also recommends they take some graduate level courses in quantitative techniques and develop a sound understanding of predictive modeling. Brown points out that data mining users include business technologists, quantitative analysts, and IT professionals, and that members of these disciplines often work together on data mining teams. "The companies that are most successful [with data mining] bring these three together from the beginning," he said. Brown adds that, "in small to medium sized companies, IT has to do it all," but in Fortune 500 companies, the IT department often provides a technical professional with data mining skills to serve as part of a data mining team. Srinivasan counsels that IT workers interested in data mining should learn to, "articulate complex data extraction queries" that will produce meaningful results. In data mining, he said, "It's not who asks the most questions, but who asks the most interesting questions."
© Copyright 1998 The Washington Post Company |
|||||||||||||||
|
|
|
||||||||||||||
|
||
 |
||
