Articles & White Papers
The Real-Time Data Warehouse: Is Data Warehousing Dead?
By Michael Haisten
Originally published in DM Review Online, in October, 2001
The king is dead! Long live the king. In old England, the proclamation that the reigning king is dead was immediately followed by the seemingly contradictory cry of "Long live the king." This was to herald the coronation of the new king demonstrating the essential continuity of the realm. It was unthinkable to proclaim the end of one regime without announcing the simultaneous arrival of the new.
In our time of mind-numbingly rapid change, an insidious form of cynicism has taken hold. We are constantly on the lookout for signs of weakness in technological trends so we can renounce a former infatuation, which is now seen as a fad that has peaked. The more seemingly successful the trend, the more vicious is our attack when reality sinks in. This accounts for the vindictiveness of our hindsight appraisal of the dot.com bubble. The king is dead! What now?
Back in 1997, the technology press in the U.K. jumped on the bleak appraisals of several broad-based studies to proclaim the death of data warehousing. In fact, they suggested it was stillborn, since it never seemed to produce the promised returns on investments in the vast majority of cases. The king is dead! What now?
In the U.S.A., where technology vendors have more sway over the media, the reaction was more muted. The vendors were unwillingly to give up a successful marketing tagline without a suitable replacement. Yet, nobody could deny the decidedly mixed results of data warehousing investments by 1998. A deluge of moderate to outrageous failures has swamped the few stellar successes. The obvious conclusion is the concept itself is a failure.
This conclusion is even more flawed than the overreaction to the dot-com crash. The wastefully extravagant dot-coms failed because they flew in the face of economic reason. On the other hand, the basis of the data warehousing concept is not only sound but evermore essential in our era of rapid change. Integration and integrity in pursuit of intelligence are the cornerstones of the information age.
The concept is not flawed it just morphed beyond all recognition. The semantics of data warehousing prompted the vast majority of implementers to design with a "storage" mentality. In the end, all they built was yet another database. Successful practitioners, on the other hand, design with a "flow" mentality. Their objective is to collect, assimilate and deliver data as efficiently as possible to where it is needed now. As in the real world, warehousing has moved from the tactics of storage optimization to the strategy of logistics.
Warehousing today is about managing the information supply chain from end to end. We reach back for raw data as it is generated and transform it on the fly (real-time data warehousing). We integrate and automate the business intelligence process to effect change continuously (closed-loop analysis). The concept now extends over the front ends (portals), delivery vehicles (data marts) and exploratory methods (multidimensional cubes) to encompass the entire range of information exchange. It ends only at the point that information facilitates action.
Look into the heart of every dominant company and you will find the same thing: a mastery of the flow of information in support of business goals. Data warehousing is dead! Long live data warehousing!
|
