Big Data and HR: when analytics become personal

We live in the Code Halos age where everything that we do, every interaction that we have- positive or negative - generates data that lives somewhere in the “cloud”. No matter your age, there is probably a digital trail that you have left behind, data about you starts generating even before you are born. In fact there are many studies that capture data about the development of a baby while still in her mother’s uterus. But what does this mean for us? If it is a good or a bad thing that all this data exists and more importantly persists as we are born and grow up? More importantly, how does this affect our ability to get a job, create a company or participate in public events? This blog will attempt to answer this question by putting it in the context of a real scenario given to me by one of my customers.

This customer is a national retail chain that operates over 8,500 stores and employs over 185,000 associates. This company HR department probably interviews hundreds of candidates a day which costs the company thousands of dollars in people’s time; not to mention that while they have been refining and maturing their interview process it is still not a 100% accurate and bad hires still get through sometimes. What if rather than an elaborate interview process which astute candidates can manipulate, the company implemented a big data system that could process all the information in existence for a particular candidate. All the records since that person got in the “system” would be made available to an engine that would recommend the right candidate(s) for a particular role. Not only that the system would be able to predict (with a high degree of accuracy), how well that person would do on the role, when that person would get promoted, identify the highest position that person would probably occupy through their career in the company and more importantly identify the likelihood of retaining that person in the company the right time to achieve his/her maximum potential.

Is this real or a product of a very imaginative science fiction mind? Before we decide to answer this, let us explore other areas that deal with the very core of what is to be human: spouse selection. Did you know that about 25% of all marriages in the USA started online? In fact there have been scientific studies that concluded that people who met online are happier than people than met a traditional way (through friends, work, etc.). I myself did not believe this until I attended an analytics conference where the chief data scientist from eHarmony gave a presentation and he explained that when you meet in person for the first time, our instincts hone-in primarily in looks which while might lead to temporary satisfaction of being with an attractive person from the opposite sex. However this physical attraction does not necessarily lead to a relationship success in the long run because of potential misalignment in key areas (e.g. career goals, way to raise the kids, etc.). He explained that the online matching industry has developed a set of personality tests that are extremely difficult for someone to fake and once this information is in their systems, they can effectively use it to populate the analytical models that will look for long term compatibility as the primary criteria for matching.

So going back to our original question if big data can provide better results in HR than a face-2-face interview, I argue that not only it can but it will. If something as complex and personal as meeting the right person that will become your spouse is now in the hands of a big data algorithm, it is just a matter of time before big data does the same for your next job interview.