The Battle for Payment at the Point of SalePrevious Article
Why Loyalty? - Immediate Health CareNext Article

Dusting Off Dirty Data

Line Spacing+- AFont Size+- Print This Article
Dusting Off Dirty Data
Related Articles

I like to joke with clients that a strategy recommendation without a foundation in customer data is not much more than a “best guess”. Even though recommendations might be highly educated and founded in experience, but they are still just a starting point with good intentions. Self deprecating humor works most of the time, but in this case, there’s quite a bit of truth below the surface.

To fortify my “best guesses”, I enlisted the services of the Boire Filler Group a few years back. The Toronto based firm specializes in data analytics and predictive modeling, and our partnership has flourished over time. In the process, I’ve learned some important practical lessons that separate those who “talk” about loyalty from those that “work” in loyalty.

As the result of work during the past two years, I have come to understand there is no such thing as a “simple customer segmentation”.  To be candid, we’ve had several prospective clients balk at the idea of spending any significant money on data analysis, asking what we can deliver that they cannot already produce. Most organizations know they have lots of data and are biased to believe it is of good quality. They naturally have resources on staff responsible for curating the data, and are hesitant to spend money in duplication of their internal efforts.

Based on a survey of projects completed over the past two years, I submit data does exist, often in high volumes, but it is quite often messy, with many data fields sparsely populated, and without ability to tell a story about customer transaction history and value. In essence, there’s lots of data, but much of it has to be described as dirty data.

The Boire Filler approach to analysis addresses this gap and begins with a data discovery process. A review is made of the data dictionary and a sample data set is evaluated for quality, consistency, and usability. Frequency reports are generated to determine which data points have high utility to support strategic insights, and it is not unusual that the set of variables used for a 3 variable segmentation departs from the expected list.

The revelation here is that the quality of a company’s data, defined as its utility for marketing purposes, is not directly related to the volume of data on hand, the length of time it has been collected, or the number of reports that are currently being generated from it. The benefits of dusting off dirty data are several fold:

  1. Approaching a data analysis project with an open mind and accepting that there may be gaps in quality is an early key to success.
  2. Investing in a data audit pays dividends and uncovers areas where the greatest progress may be made.
  3. The results of the data discovery process and eventual segmentation analysis provides a framework for improvement in future data collection procedures, improving the quality of the data asset over time.

With all the talk about data being the new oil, there should be equal air time to the fact that we need to increase our “refinery” capacity. Engaging a data discovery and audit process is the first step in clearing the pipes to let the “the new oil” flow.


Featured Posts

Customer Loyalty and Natural Disasters

Customer Loyalty and Natural Disasters

Living in a Tropical climate zone has its risks. South Florida is positioned as a peninsular-shaped bowling pin, and each year between June and November, an average of 11.3 storms [...]

Become a Loyalty Truth Insider!

We don't publish often to our subscriber lists, but when we's powerful information that you can use in your business. Just share a few bits of information here and consider yourself to be an insider at
  • This field is for validation purposes and should be left unchanged.

2nd Annual Loyalty Academy Conference – Coming in March 2017! Sign up now for early bird pricing!

FREE Loyalty Manifesto

Event Calendar for Loyalty Marketers

Pin It on Pinterest

Share This