Experts: Technology Is Not To Blame For Data Warehouse Failures
Despite reported failure rates as high as 90 percent for data warehousing projects, a panel of experts here said the technology is sound and that many common problems associated with such projects can be avoided.
The panelists attributed many of the failures to corporate culture and other human factors that conspire to hinder or stop data warehousing projects. The remarks came as part of a session entitled “Preventing a Data Warehouse Catastrophe,” during the TechDec Exposition and Conference held here from October 7 through October 9.
A data warehouse is company-wide database that brings together data from disparate units or operations to support marketing and other decision-making in an organization. When the database is organized for a single business unit within an organization, it may be referred to as a “data mart.”
“Despite the high failure rates for data warehousing, people still want to do it,” stated panelist Seth Rachlin, co-founder, president and CEO of New York-based Connect Systems, Inc.
“Data warehousing can deliver significant return on investment across the enterprise,” said Rachlin. “This is true in many industries and it is particularly true in insurance. I still haven’t seen the insurance sector embrace these kinds of technologies the way industries like publishing and telecommunications have,” he added, “but there’s certainly a lot of interest there.”
Rachlin also maintained that data warehousing technologies have matured significantly. “The technologies out there are pretty good,” he observed. “It’s been out for a while and people know how to do it.”
He added that recent development of Web-hosted technologies have made data warehousing accessible to more end users. “From a technology standpoint,” said Rachlin, “implementing a data warehouse is not that difficult.”
According to Rachlin, the principal reason so many data warehousing projects still fail is that “companies are not good at running projects at an enterprise level.
“Particularly within the insurance sector, there is a product or functional focus versus an enterprise perspective,” he continued. “They understand everything there is to know about a particular product they’re selling. They see themselves as completely tied to that product.”
Another reason for data warehouse development failures is that different business units within the same company will have different definitions of common terms, such as “premium” and “agent,” said Rachlin. This complicates the process of working with data from the various business units.
Executives can also get territorial when it comes to ownership of the data between business units, Rachlin noted. Claims of ownership may also extend to distribution networks and to the customers themselves.
Another consideration, according to Rachlin, is that while “data warehouses do great things, they also unsettle people and make them nervous.” The creation of a data warehouse can change the way a company typically does business, he noted, and as a result some shifts in corporate power may take place.
“It does threaten people,” said Rachlin of the data warehouse concept. He added that those who take on data warehouse projects “need to recognize that not everyone is your friend.”
Rachlin also pointed to the challenge of setting priorities in terms of what gets done first in a data warehousing project. Having a method and process to determine those priorities is “very critical,” he stated.
What can be done to overcome some of these obstacles and meet the challenges? Rachlin said getting high-level commitment within an organization is a key factor, along with encouraging broad participation in the project within the organization. “You can’t do this kind of thing on the stealth,” he observed.
One way of encouraging participation is to regularly communicate the status and progress of the data warehousing project to everyone concerned, said Rachlin. He pointed out that “cleaning up hundreds of years of data problems” and setting up the project takes time, energy and effort. This part of the project “doesn’t appear to have immediate results,” so it is important to communicate frequently, he added.
Successfully developing a data warehouse involves “changing the culture, not just the systems,” said Rachlin. There is a need to “broker consensus” from all those involved, particularly in terms of “what things mean and how they will be reconciled,” he continued.
Getting consensus doesn’t necessarily mean “someone wins,” Rachlin noted. “It may mean you have to tolerate and live with four different definitions of ‘premium.’”
Rachlin also recommended “federated ownership and governance” of data warehouse projects, rather than having one small group in control.
Yet another way to help ensure success is to “tie funding to specific deliverables, and to specific lines of business,” said Rachlin. “That’s the surest way to make sure [budgets] are not cut.”
Another panelist, Mike Heilman a business technology consultant at Nationwide Insurance Company, Columbus, Ohio, agreed on the importance of finding an “executive sponsor” for a data warehousing project. “If you don’t get that commitment, you will not succeed,” he asserted.
Heilman also agreed that establishing data definitions is a key to success in data warehousing projects, as well as aligning the project’s goals with the company’s key business drivers, such as profitability.
In addition, Heilman emphasized that “project management skills are the single most important technology resource that comes to play on data warehouse development.” Such skills involve management of technology, people and processes, he explained. “You must balance all three. Any failure in an area means failure of the project.”
Panelist Bill Sinn, director of financial services marketing for Dayton, Ohio-based Teradata, provided a laundry list of reasons for data warehousing project failures. These included: the project being too overwhelming; difficulty in quantifying return on investment; lack of IT proficiency with data warehousing initiatives; the scope being too limited; poor data quality; the project being too expensive, and not having sufficient time.
The key to overcoming such challenges is to commit to necessary changes, said Sinn. “Technology is usually the scapegoat [for project failure], but the technology is proven,” he noted. “It’s really about culture change. It’s pretty threatening to have your job changed.”
Sinn called on those who would manage data warehousing projects to be “champions” of change.
“Data warehousing is an instrument of change, but not an engine of change,” said Sinn. “People are the drivers.”
Reproduced from National Underwriter Life & Health/Financial Services Edition, November 5, 2001. Copyright 2001 by The National Underwriter Company in the serial publication. All rights reserved.Copyright in this article as an independent work may be held by the author.