DECOODA
10x'ing data quantity
and quality for Decooda
Decooda helps businesses understand why customers do what they do, surfacing helpful insights from unstructured data like reviews, support requests, and in-product behavior.
UNDERSTANDING CUSTOMERS
Supporting smarter investment decisions
Decooda helps businesses understand the “why” behind customer behavior, but a critical piece
of their AI platform was starving for good data.
Powering that platform is a small army of ML models and bespoke classifiers that extract information
from customer activity, including reviews and in-product actions.
Given the granularity and accuracy of Decooda's platform, some of their customers apply these tools
to market analysis, using the resulting insights to inform investment and financial decisions.
THE KEY INGREDIENT
Getting ML systems
the data they need
To better serve their customers, Decooda supported this with an additional suite of
purpose-built models. But these finance-focused models were lacking the most important ingredient in a modern AI system: large volumes of high quality data.
They had built a tool for labeling their troves of unstructured data, but the labels were noisy
and the people using the tool were taking longer than expected to label each item.
Our first step in working together was to do an audit of their existing system and
identify the least effort, highest value ways to improve it.
UX + ML
Eliminating choice paralysis,
facilitating flow
Combining our experience in design and machine learning, we identified user experience as
the primary limiting factor. Users were faced with over a hundred potential labels for each
item, leading to overwhelm and analysis paralysis.
We consolidated related labels and streamlined the user interface, resulting in a speedup of 9.3x for labeling, while sacrificing only a small amount of granularity.
This improved UX allowed users to enter a flow state while labeling, turning it from
grueling drudgery into, if not a fun activity, at least a satisfying one.
TRUSTWORTHY DATA
A new standard
for data quality
With our UX improvements unlocking a new abundance of data, we could afford to dramatically
raise our standards for quality.
We worked with the team to implement automatic data quality checks and flag unreliable
users, updated the system to only consider labels added by three or more users, and expanded
the app to support on-demand crowdsourced annotations from the AWS Mechanical Turk service.
A SUCCESSFUL EXIT
Tripling prediction
accuracy
The resulting data powers a broad swath of Decooda's finance-focused models and classifiers, and on average has tripled their precision and recall scores. In part on the strength of these models, Decooda was recently acquired, joining forces with North Highland, a leading change and transformation consultancy.
