The Differences Between AutoML and Market Simulation

By John Pasinski, October 18, 2019

Automated Machine Learning, commonly abbreviated AutoML, has emerged as a solution for organizations that are looking to enable their teams to use machine learning more efficiently. This approach has set up organizations to capitalize on their available data to address a variety of tactical business questions, such as which specific customers are most at risk of attrition or which products are most likely to appeal to certain individuals. AutoML works by automating the workflow of data processing and machine learning model exploration and tuning. This enables teams to construct models more quickly and leverage large, structured datasets.


Market simulation is a distinct approach that helps organizations test what-if strategies. It complements AutoML by addressing business questions in contexts where machine learning’s power is constrained, such as when data is limited, dynamics are shifting, interactions are complex, or when competitive and external market forces are at play. Market simulation starts with domain expertise, and is used to recreate complex real-world dynamics. It leverages both structured data and qualitative insights, allowing teams to explore a broader scope of interactions.

Every modeling approach has its tradeoffs.  Our customers frequently ask us what the differences are between AutoML and market simulation.  Here are four specific areas to consider which approach is right for you:


1. Data and Expertise Requirements


For AutoML to be effective, teams must leverage large datasets with thousands, millions, or more data points. AutoML then helps identify patterns within these large datasets. While domain knowledge may help in interpreting and validating model results, it is not required to achieve an answer.


Market simulation, on the other hand, requires a theory of how a given market works. This means that domain knowledge is very important at the start. This provides the opportunity for business stakeholders to get involved early on in the process and ultimately helps them feel more confident in the results, knowing that it accounts for their human experience.

2. Interpretability


AutoML leverages large amounts of data to construct relationships using complex algorithms. While its ability to uncover relationships buried deep in large datasets is a strength, the nature of those relationships is often not transparent. Understanding the “why” can be a substantial challenge with AutoML. Typically, data scientists need to put much effort into extracting and translating the relationships uncovered by models to business users.

Market simulation constructs and models out complex dynamics and interactions. Through the calibration process, it becomes clear which levers in the market are driving outcomes and those can be visualized using a number of diagnostics, such as sensitivity analyses and attribution analyses.

3. Shifting Consumer Dynamics


With the right data, AutoML can help teams identify answers to specific tactical questions around consumer preferences. For example, an algorithm could be devised to rank which products are most likely to resonate with a specific customer given certain attributes. This approach works well so long as the underlying preferences are static. In other words, it works if the status quo holds. This makes AutoML useful for managing short-term, tactical decisions.

Market simulation works well in helping organizations navigate shifting consumer preferences. For example, the same group of consumers may behave very differently before, during, and after a brand campaign, recession or PR crisis. Because market simulation incorporates data or insights on consumer perceptions and drivers of choice, it is capable of identifying how various events and interactions influence the evolution of consumer preferences over time. Market simulation helps teams navigate strategic choices when the economic environment or consumer trends are shifting. It is also useful when conducting what-if forecasting on changes that may be coming in the future.

4. Competition


AutoML is useful for modeling out behavioral patterns for specific individual consumers. In this way, it can be helpful in answering questions like which consumers are most at risk of attrition or which TV program is a specific individual most likely to watch. So AutoML can guide tactics around items like customer retention or personalization.

Market simulation is useful for understanding the behavior of a population, and modeling out how consumers choose among competing alternatives. Market simulation allows for teams to put their brand in the context of competitors and run what-if forecasts to answer strategic questions. This capability empowers teams to tackle competitive market strategy, including questions like: How will our business strategy impact our market share and that of our competitors? How would a change in competitor strategy influence our market outcomes? What response to a competitive shift is most likely to benefit our brand?

Managing the Forest and the Trees

A helpful analogy when comparing strategic planning to tactical execution is thinking of the forest (the broader business strategy) and the trees (specific tactical questions). Market simulation is helping organizations leverage their data and expertise in new ways to inform strategic decision making through what-if scenario testing. AutoML complements market simulation enabled strategy by providing tactical insights for specific problems where lots of structured data is available. With the combined two distinct approaches, teams have a toolset that allows them to effectively manage the forest and the trees.