Customer Data & Decisions – “Big Data – Big Waste?”

Part 1 was about what firms really do know about me as a customer. Part 2 covers the ever popular topic of Big Data and why it needs a sponsor, action plan, and a solid analytic platform.

What did you say?

Did I get your attention with my somewhat controversial headline? Maybe it’s actually not that contentious because simply wiring to and capturing lots of data (e.g., Big Data) does nothing for you except add cost if you don’t effectively glean insight from it, and take action on it.   It’s no different than any other asset, in that if it’s idle, it’s sucking energy and not providing any return.   Like a big data black hole, where data enters but no insight can escape. How do you combat that?


Have an action plan based on the kinds of customer decisions you want to improve, investigate data required, and constantly test, monitor, and refine that plan.   This plan will dictate what data you should be seeking and exactly how you will leverage it. In other words, work backward from your desired outcomes.

You might also be asking, what is big data? Good question.   As a participant in the business intelligence revolution, I’ve seen massive databases used for years for decision purposes. So what is new and different?   Actually, there are a few things.

First, customer data has been historically captured, scrubbed, matched and restored into on premise structured databases.   This led to the enterprise data warehouse with the so called “360 degree view” of the customer.   These systems required data expert intervention to add new data elements, were usually on premise, and latency rendered the view stale for today’s standards. Consumer and market expectations have evolved to expect on-demand and streaming data reflecting the latest and greatest view of the customer.

Second, since it ultimately required a target structured store, unstructured data, which is massive, became difficult to assimilate into one structured data warehouse.

And third, the variety of structured and unstructured data sources have grown, so much that again using an approach of trying to codify and blend all of that data into one mart did not meet flexibility, agility, and timing requirements of business people trying to make better decisions.

Ok, I need a plan. What next?

What if you could identify and sway vocal and influential customers? What if you could proactively identify customers at risk, and take actions to not only save the relationship, but turn them into ardent supporters?

Take these types of questions, and work backwards to formulate your plan.   Call it your big data blueprint.


Do you already know who the most influential customers are?   If not, start there.   How would you define this?   Conventional wisdom may first suggest it’s those with the biggest network of followers or highest NPS score. But upon further review, what might be more important is customers that actually frequently refer versus ones who say they will.   Working back, you would need data like mentions and referral codes. So determine the particular outcome, and then concentrate on connecting to the data you need to monitor and track those actions – viral actions such as re-tweets, re-posts, forwarded links, reference events, and such.   Then, rate your customers on that basis – building a Clout Score – the higher the score, the more clout they have with others and the more they refer you.   This score is then connected with actual behavior instead of formulations, surveys, or postulates.

Likewise, figuring out which customers are at risk, you might hypothesize that a major service interruption would put them at severe risk, and thus simply being able to run a query to find all customers impacted by key service disruption events would suffice.   Yet often, customer retention risk is much more complicated than that, and it’s likely that in this case you need a behavioral model that considers various risk factors, such as service disruption patterns, social sentiment, clout, customer loyalty, competitive options, and switching costs – and then test that model against real churn outcomes to calibrate its effectiveness.

Having sponsors is vital because invariably some aspects of your big data plan will involve capturing and leveraging data not readily available, and thus sustaining funding and resources to see your project through will require champions – people who believe strongly in the cause, and can help.

What technologies can help me get to my happy place?

We don’t live in a simple world.   We accept that, or get lost, frustrated, and fall behind – but we do expect technology will continue to help us navigate the intricate world.   So we seek the simplest and fastest solutions to complex problems.

Your answer is you will need many technologies. Accept that and do business on that basis. Select your stack based on requirements that your vendors are open, constantly invest in innovating their underlying technology, have exceptional integration both with their own sister products and with the outside world. Consider firms with a robust ecosystem and strong reputation for training, partners, and professional services.

Big data systems involve storage and retrieval of unstructured information, which is data that has not been highly codified from its raw form.   For example, data entered into free form text such as comments to blog posts or data collected from digital activity such as granular website click activity.   Big data is also real-time streaming data coming from various sensors that are always on, and stream data (often 24×7) such as devices that report precise location of objects (e.g., mobile phones).

Partner with a vendor that has solid, modern, and open technology, and has it in one platform. Beware of companies that get the marketing right and have compelling messages, slides, and even case studies, but under the covers have 2 or more actual platforms stitched together, and requires more custom coding to meet your requirements. How ironic that I’m warning marketers to be wary of the marketing!

Comments and alternative views are always welcomed.

Note:  These views are my own, and not that of my employer