Don’t Be Afraid of Poor Analytics

If you’ve taken any analytics or statistics courses, you’ll have been bombarded with cautions about how analytics can go wrong. For example:

  • Random outcomes appear meaningful to the untrained eye.
  • Small sample sizes give misleading results.
  • Without good control groups, you can’t be sure what caused an outcome.
  • Correlation is not causation.

We could go on at some length. In fact, sometimes it feels like statistics is all about showing that whatever analysis you did, your conclusions are sadly mistaken.

In the world of business analytics, we shouldn’t be afraid of poor analytics. The reasons are straightforward:

We need to start somewhere. If we discourage beginners from doing analytics, they will never become experts.

Some analytics is, more often than not, better than no analytics. Let’s imagine a restaurant needs to know if their chef’s special is any good. They ask the first three customers who try it. One likes it and the other two feel it’s mediocre. That’s a small sample size (n=3) but it’s all we’ve got. It could be leading us to the wrong conclusion, but it’s still a sound business decision to take the special off the menu rather than risk alienating customers.

These two reasons are not incidental. It is hard for people to do any analytics so if we allow barriers to get in the way they will give up. Furthermore, it is very rare in business to have clean, randomized control groups with large sample sizes. If we set a high bar, we will not have better analytics, we’ll have no analytics.

Article Continues Below

I hesitate to say all of this because, of course, we want people to do as high-quality an analysis as possible. We don’t want sloppy work nor do we want them to be blind to the weaknesses of their analyses. I hope I’m not seen as making an argument for poor work; I’m trying to make an argument for doing the best work possible given limited time, data and expertise.

Peter Navin, the SVP of employee experience at Grand Rounds and author of The CMO of People says, “Measure what you want to become.” By that, he means it’s better to give a poor measure of something important rather than a great measure of something trivial.

If the restaurant had a dashboard, they should have a place to report on the quality of the chef’s special. Their existing measure may not be great, but if it’s important, then let’s work with some data rather than no data; and let’s aspire to the analytics level that we want to become.

David Creelman, CEO of Creelman Research, is a globally recognized thinker on people analytics and talent management. Some of his more interesting projects included:

  • Conducted workshops around the world on the practical aspects of people analytics
  • Took business leaders from Japan’s Recruit Co. on a tour of US tech companies (Recruit eventually bought for $1 billion)
  • Studied the relationship between Boards and HR (won Walker Award)
  • Spoke at the World Bank in Paris on HR reporting
  • Co-authored Lead the Work: Navigating a world beyond employment with John Boudreau and Ravin Jesuthasan. The book was endorsed by the CHROs of IBM, LinkedIn and Starbucks.
  • Worked with Dr. Wanda Wallace on “Leading when you are not the expert” which topped the “Most Popular List” on the Harvard Business Review’s blog.
  • Worked with Dr. Henry Mintzberg on peer coaching, David’s learning modules are among the most popular topics.

Currently David is helping organizations to get on-track with people analytics.

This work led to him being made a Fellow for the Centre of Evidence-based Management (Netherlands) for his contributions to the field.



Leave a Comment

Your email address will not be published. Required fields are marked *