Businesses don’t do data analytics because they are curious, they do it because the want results: to find levers in their own behavior that will change the outcomes in their favor. Yet most organizations aren’t exploiting the data they have to the fullest; their data-driven culture is still plots-driven culture — data generates plots, and people decide based on those plots — which means they lose money and time in efforts that less intuitive forms of analysis would have ruled out, and sometimes miss opportunities they would have suggested. The following is an example of both the problem and the solution, based not on observed data (most data of this kind is too valuable to be public) but rather generated by a simulation engine, and leveraging a private tool I developed to streamline my work on this kind of project.
Stop me if you’ve heard this one…
Your online marketing team has good data showing that Twitter reach influences positively the sale of your product (let’s say, movie tickets):
What’s better, larger Twitter advertising budgets are correlated with a larger Twitter reach:
Businesses rarely get a clearer opportunity than this. You might or might not have both sets of data in the same database to be processed by the same team, but those two slides make a convincing, data-driven case for increasing your budget for Twitter advertising. Yet after you follow thru with the idea, you’ll probably find a disappointing result, as predicted by the lack of correlation between Twitter advertising budgets (the only thing you’re in control of) and movie ticket sales (the only thing you care about) in data observed after implementing your new strategy:
Embarrassment abounds, your data scientists unhelpfully explain that correlation isn’t transitive, and faith in the use of algorithmic analysis for strategic decision-making takes a dive.
So when next shown a couple of slides showing that Instagram reach is correlated with ticket sales
And that Instagram advertising budgets are correlated with Instagram reach
Most organizations will, understandably, feel skeptical about the prospect of placing yet another bet without a clue of what might happen.
… or at least make a more educated guess. The last years have seen the full maturity of a set of mathematical and software tools designed specifically to build causal understandings of the world, not just of what’s correlated with what, but also of what, when changed, will change something else, which is precisely what decision-makers need, both to reduce avoidable errors and to find unexpected opportunities.
The thing to keep in mind is that these algorithms don’t think like humans. They aren’t “intelligent” in the sense of a robot in a sci-fi movie, but the thinking they actually do, the use of data to find causal relationships, they do in ways that sometimes feel obvious and intuitive to humans, but, most importantly, sometimes doesn’t. What’s the point, after all, of a program or an employee who never has anything surprising to contribute?
Here’s the result of applying one of those algorithmic analysis tools to the data shown in the plots above, and only that data, showing what, in the opinion of the algorithm, can be used as a lever to influence what:
As you can see
- The algorithm picks correctly which advertising budgets impact which social network; it doesn’t read the variable names like we do, so it was a deduction purely based on the data itself.
- The algorithm, weirdly, thinks that Instagram reach influences Twitter reach. This could be true, or could indicate a missing common cause – algorithms aren’t omniscient, just very good at telling you what the data says.
- The algorithm is quite certain that Twitter reach does not influence ticket sales. Had you run it at the beginning, it would have recommended against increasing the Twitter advertising budget, and would have prevented that wasted expense.
- The algorithm believes, on the other hand, that Instagram reach does influence ticket sales. The plots would have been equally convincing (or, after the Twitter fiasco, unconvincing) to a human, but the mathematical analysis of the data finds a potential effect through Instagram that Twitter, again according to the data, doesn’t have.
And, indeed, if you increase your Instagram advertising budget, ticket sales do go up:
It’s important to emphasize that this isn’t an artifact of the data set I generated, but rather of the rules of the simulation engine with which I generated them. In other words, the algorithm correctly inferred some of the rules for obtaining better outcomes in that simulated world, just as, applied to data measured in the real world, it correctly infers some of the rules for obtaining better outcomes there, in whatever business context you’re in.
The digital advertising industry refers to the problem we just dealt with as the attribution problem, but it’s at the core of applied data analysis in every industry and context: how do we leverage the numbers under our control to change the numbers we care about? The takeaways are pretty much universal:
- Without an understanding of what’s actually going on in the world, even the clearest data can lead you to a dead end, or worse.
- Either your data scientists have to have an understanding of the domain they are analyzing, or you should pair them with people who do.
- But also, and this is relatively new, you can analyze the data you have to build this understanding, which comes specially handy in contexts that are new, specialized, or whenever you just want an extra advantage against competitors who have access to the same expertise you do.
It’s a common experience for companies to first go through a brief honeymoon with data science as they can finally see both detailed data and high-level patterns they hadn’t been aware of, but soon the rush of omniscience is replaced by the frustration of being able to see what’s going on, but not to change it. The kind of analysis I’ve shown here is one of the newest and most effective ways to discard at once many of the initiatives that the data — not in a way obvious to humans, but very much so to the right algorithms — already predict won’t work, and to prioritize the actions with a better chance of influencing the outcomes you care about.