Is real time, the right time?

With the advent of the smart phones, it seems that the next wave of BI will come from “mobilizing” the business applications and putting real time information at the ever reach of the executives and decision makers. However, experience dictates that having information available in real time is not always the best option. Early in my career, when we started experimenting with real time data loads, primarily to get around the limitation of fixed time load windows, I found out (the hard way) that sometimes real time data might give more trouble than benefits. I had the opportunity to witness over an upgrade from daily loads to 5 minute batches when I was working for a retailer early in my career. The new system enabled the data warehouse to be updated from all the stores 12 times an hour. From an IT perspective it was a huge success, being able to load data from more than 200 stores in less than 300 seconds from the POS to the Data Warehouse. However, our enthusiasm was short lived when we tried our business users to acknowledge the value of this data. The feedback started coming in that our users could not make any decisions as it was hard for them to tell exactly what was on the data as there were intermittent delays in some of the stores (links/servers down, etc) that prevented from them knowing exactly what they were looking at, and because the data changed every 5 minutes, it became a very troubling exercise to catch if there were any data errors (either introduced at the source or as part of the loading process). It got so bad, that our users asked us to remove the capability and give them access only to the audited data (as of close of business yesterday). For me this was a very valuable lesson that not always faster is better. Our brains are designed to process information when we understand what we are looking at; after analyzing the data you might want to run different slices of analysis which at the end of the day you want to amount to the whole pie.
Imagine that you are driving a next generation vehicle where everything is electronic and it can communicate with your Smartphone in real time. It can give information to the second on gas levels, air pressure, oil life, etc. If the car was sending information to you every 5 minutes with the different metrics speed, avg. fuel consumption, miles left to empty,etc, you would probably start ignoring the data because it becomes non-relevant after a while. However, with this comes the added risk that if you have enough fuel for 2 miles, you will probably miss this tidbit of information among all the other data elements. You would have run out of gas and not for lack of information, but rather excess of it.
Compare this situation with the concept of right time: you get only key notifications with the right amount of notice so you can act on it, for example the car will only SMS you when there are 2 gallons of fuel left, giving you enough time to locate a gas station and refuel. As we witnessed the next mobile evolution (or revolution) in BI, we need to able to separate the fads from long term trends. If you are caught on the real time frenzied, take a deep breath and ask yourself: “Is real time the right time?“