Saturday, August 16, 2014

The price of awareness

The price of awareness

Today’s blog while a bit dramatic and spiced with some movie references will attempt to highlight the correlation between the pursuit of knowledge, the challenges that we have to overcome to achieve this enlightenment and more importantly the moral and ethical obligations that we contract once we achieve this knowledge.

The human race is defined by its curiosity and the motivation to learn more: from our humble beginnings on Earth, we have been able to achieve a highly technological and evolving society. These inherent personality traits drive entrepreneurship and business expansion from the CEO to the entry-level clerk in an organization.

If we take a typical Fortune 500 company and we apply this logic, we will find numerous individuals that work every day to improve their day-2-day. In order to do this, these individuals must make a significant number of decisions and there is no better way to make these decisions than by having the right information.

However, getting the right information is typically not easy. There are multiple systems and thus multiple sources, not to mention that there are many interests in the organization that might influence how the data is captured, stored and reported. In order to address these challenges, historically organizations have started large Decision Support System programs, typically with a Data Warehouse strategy at its core. However, the implementation of these programs took a long time and it was always a very risky enterprise, not necessarily because of technology but due to the fact that the effort challenged the status quo where particular individuals or team where the guardians of information. In spite of these challenges, many companies prevailed and they were able to successfully implement Enterprise Business Intelligence systems. However, not many of these systems were able to deliver the value that it was promised. In fact quite a large number of implementations was left without use after the initial pilot or testing.

A group of researchers from a prestigious consulting and delivery firm started looking into this and they realized that in many of these situations the systems were working properly and the information was being reported with extreme accuracy but people still refused to use the system. When they explored a little deeper, they found two some surprising things:

a)    The new system had different results for some of the key metrics that the organization was using in the past

b)   The organization decision makers were getting access to new information that they did not have access before

The researchers were puzzled, as the results seemed to indicate that the new programs had been extremely successful both correcting past mistakes and providing light to previously dark areas.

However, what they failed to consider initially was the price of awareness. On the surface, while it looked that the new system had more accurate values for key metrics, they found out that some people had made their entire career in the organization reporting the old values and sometimes these values had been reported to top management and Wall Street itself. Publicly acknowledging that the old values were incorrect would have caused a severe reputation hit to the individuals who built their careers using the old ways not to mention a possible catastrophic stock drop for the organization.

In regards to the second situation - where people had new data available that they did had before, they also found that these new data insights were holding people accountable to new standards and as such they both limited the amount of freedom the decision makers used to have before the new systems became available and it was also binding them to perform an action or set actions based on the new information available.

So in a way it was like in the matrix when Neo was offered two pills: red (to start a new path on the search for knowledge) or blue (to forget and go back to his old life). While Neo took the red pill and he never regretted the decision, there were other individuals who did not have the same strength of character and got to the point of betraying the entire Human race for a false hope of going back to the bliss of ignorance.

There is no denying it, with new information comes new power and with new power comes new responsibility. There are moral, ethical and sometimes-financial implications that are implicitly accepted when we leave our ignorance behind and choose to see the truth as it is. However, history has proved that we will always be better off if act according to the right, truthful information and as inconvenient as it can to be “ahead of your time” it will always pay off to do the right thing.

If you are in this situation, don’t be afraid to take a leap of faith and trust that everything will eventually be all right. After all, there is no greater legacy that you can leave behind than preparing yourself/your Team/your Company to face the future with the right tools and perspective having started the change while there was still time left in the clock.


Monday, April 21, 2014

On vacation with Big Data


According to recent statistics more than 90% of people in the USA spend their days within 30 miles of their home location. They define their daily circles around things they know and are familiar to them: their neighborhood, schools, shops, etc. We are all creatures of habit: chances are that what you do on a typically Monday will be repeated week after week with little variation until you go on vacation. Vacations are truly events of consequence because it is one of the few occasions where people disrupt their routine, it is quite common while on vacation for people to travel more than 30 miles from their home location, either by car, planes, boat or any other transportation method. While on vacation a person is away from the familiar and he/she is probably more open to visit new places, try new food, meet new people or just wander around.

If you think about, this is the perfect opportunity for an external party to influence you, your guard is relaxed and you want to have fun and experience new things. Hospitality companies are aware of this and are starting to use Big Data technologies to make sure you have the “best experience away from home” (alas it does not hurt if this “best experience” puts more money in the hospitality company pocket). Let us explore how Big Data is directing your experience.

Big data influence in your trip starts way before you book your trip. In fact, most hospitality companies today use advanced analytics to drive their marketing campaigns to match their offering/properties to customers/prospects interests and preferences. There is a probably a digital trail that you have created in any (and maybe all) of your previous trips. This digital cookie gives enough information to the hospitality companies to target you on a way where you are most likely to respond.

Once you have decided to stay at a particular property and book the trip, Big Data technologies are used to present you with “bundles” or “packages” that enhance your experience. It can be a combination of car plus your room, or include meals or events. The beauty of Big Data is that regardless if you select, ignore or reject the offering the hospitality industry keeps learning more and more about you and how you react to specific offerings.

When you check-in into the property, the Big Data analytics engine is right there with you; depending on your history with the company, management might offer you an upgrade or just recommend things for you to do around. Keep in mind that the hospitality company will try to entice you to stay longer at the property and consume their services rather than leaving the premises and risk you spending your hard earned dollars with the competition.

So next time you go on vacation, don’t be surprised if you find yourself trying a new experience that you would have never tried at home. After all you are on vacation with Big Data and it is up to you if you bring it back home with you.

Tuesday, April 8, 2014

Big Data and HR: when analytics become personal


We live in the Code Halos age where everything that we do, every interaction that we have- positive or negative - generates data that lives somewhere in the “cloud”. No matter your age, there is probably a digital trail that you have left behind, data about you starts generating even before you are born. In fact there are many studies that capture data about the development of a baby while still in her mother’s uterus. But what does this mean for us? If it is a good or a bad thing that all this data exists and more importantly persists as we are born and grow up? More importantly, how does this affect our ability to get a job, create a company or participate in public events? This blog will attempt to answer this question by putting it in the context of a real scenario given to me by one of my customers.

This customer is a national retail chain that operates over 8,500 stores and employs over 185,000 associates. This company HR department probably interviews hundreds of candidates a day which costs the company thousands of dollars in people’s time; not to mention that while they have been refining and maturing their interview process it is still not a 100% accurate and bad hires still get through sometimes. What if rather than an elaborate interview process which astute candidates can manipulate, the company implemented a big data system that could process all the information in existence for a particular candidate. All the records since that person got in the “system” would be made available to an engine that would recommend the right candidate(s) for a particular role. Not only that the system would be able to predict (with a high degree of accuracy), how well that person would do on the role, when that person would get promoted, identify the highest position that person would probably occupy through their career in the company and more importantly identify the likelihood of retaining that person in the company the right time to achieve his/her maximum potential.

Is this real or a product of a very imaginative science fiction mind? Before we decide to answer this, let us explore other areas that deal with the very core of what is to be human: spouse selection. Did you know that about 25% of all marriages in the USA started online? In fact there have been scientific studies that concluded that people who met online are happier than people than met a traditional way (through friends, work, etc.). I myself did not believe this until I attended an analytics conference where the chief data scientist from eHarmony gave a presentation and he explained that when you meet in person for the first time, our instincts hone-in primarily in looks which while might lead to temporary satisfaction of being with an attractive person from the opposite sex. However this physical attraction does not necessarily lead to a relationship success in the long run because of potential misalignment in key areas (e.g. career goals, way to raise the kids, etc.). He explained that the online matching industry has developed a set of personality tests that are extremely difficult for someone to fake and once this information is in their systems, they can effectively use it to populate the analytical models that will look for long term compatibility as the primary criteria for matching.

So going back to our original question if big data can provide better results in HR than a face-2-face interview, I argue that not only it can but it will. If something as complex and personal as meeting the right person that will become your spouse is now in the hands of a big data algorithm, it is just a matter of time before big data does the same for your next job interview.