Tuesday, April 8, 2014

Big Data and HR: when analytics become personal

We live in the Code Halos age where everything that we do, every interaction that we have- positive or negative - generates data that lives somewhere in the “cloud”. No matter your age, there is probably a digital trail that you have left behind, data about you starts generating even before you are born. In fact there are many studies that capture data about the development of a baby while still in her mother’s uterus. But what does this mean for us? If it is a good or a bad thing that all this data exists and more importantly persists as we are born and grow up? More importantly, how does this affect our ability to get a job, create a company or participate in public events? This blog will attempt to answer this question by putting it in the context of a real scenario given to me by one of my customers.

This customer is a national retail chain that operates over 8,500 stores and employs over 185,000 associates. This company HR department probably interviews hundreds of candidates a day which costs the company thousands of dollars in people’s time; not to mention that while they have been refining and maturing their interview process it is still not a 100% accurate and bad hires still get through sometimes. What if rather than an elaborate interview process which astute candidates can manipulate, the company implemented a big data system that could process all the information in existence for a particular candidate. All the records since that person got in the “system” would be made available to an engine that would recommend the right candidate(s) for a particular role. Not only that the system would be able to predict (with a high degree of accuracy), how well that person would do on the role, when that person would get promoted, identify the highest position that person would probably occupy through their career in the company and more importantly identify the likelihood of retaining that person in the company the right time to achieve his/her maximum potential.

Is this real or a product of a very imaginative science fiction mind? Before we decide to answer this, let us explore other areas that deal with the very core of what is to be human: spouse selection. Did you know that about 25% of all marriages in the USA started online? In fact there have been scientific studies that concluded that people who met online are happier than people than met a traditional way (through friends, work, etc.). I myself did not believe this until I attended an analytics conference where the chief data scientist from eHarmony gave a presentation and he explained that when you meet in person for the first time, our instincts hone-in primarily in looks which while might lead to temporary satisfaction of being with an attractive person from the opposite sex. However this physical attraction does not necessarily lead to a relationship success in the long run because of potential misalignment in key areas (e.g. career goals, way to raise the kids, etc.). He explained that the online matching industry has developed a set of personality tests that are extremely difficult for someone to fake and once this information is in their systems, they can effectively use it to populate the analytical models that will look for long term compatibility as the primary criteria for matching.

So going back to our original question if big data can provide better results in HR than a face-2-face interview, I argue that not only it can but it will. If something as complex and personal as meeting the right person that will become your spouse is now in the hands of a big data algorithm, it is just a matter of time before big data does the same for your next job interview.

1 comment:

abilash said...


I went through the blog, it was very interesting idea to use big data for HR interviews. But there are few points we need to consider even if we take Eharmony model. I strongly feel why Eharmony had became success was because as generation Y is computer addicts comparing to earlier generations when they strongly went by their intuition and common sense. this generation had distanced itself from its own species and are comfortable within their own created loom. that is dangerous .

there are 2 distincts features that we need to understand here, there is still a large amount of human intelligence even after eharmony's predictive Analytical engines Algorithms. There would be good amount of rejections been done before finalizing your spouse or move on with a date we need to get into those fact how many was been shortlisted and how many materialized and how many are still happy. The digital world is inevitable, my argument is the datas are mis interpreted, they are bringing like minded people together in a common platform just because its a trend and that is cool more than that I do not see any thing.

in the same note we already have multiple job boards were we are reaching out to active and passive candidates and Iam sure his digital data will be / is been looked by recruiters like his facebook profile his groups affiliations and his interests ets.. ,more over machines will not make a judgement beyond patterns.

let me know whats in your mind.