The Experiment Dashboard Is the Heart of Your CRO Program

Your decisions are only as good as your data.

One of the key elements of running a scalable experimentation program is having a flexible, comprehensive, and above all accurate dashboard.

Flexible means it covers metrics and segments relevant to a variety of experiments and audiences - like front-end, back-end, or experiments aimed at new or existing users.

Comprehensive means it captures the full range of experimental data by variant, for example:

• % distribution of users exposed, engaged, and converted;
• full funnel of engagement and conversion activity;
• guardrail metrics (like load speed metrics);
• 2nd order effects (like customer complaints or NPS);
• retention, churn, or reengagement metrics.

And accurate requires relentless focus on data quality throughout the experiment.

• Does the aggregate of experimental data match up with broader business metrics?
• Are there any unexplained changes in historical reporting?

Additionally, an experiment dashboard is different than a business dashboard in that it focus on capturing and highlighting differences in key metrics against a benchmark segment (typically a control cell). This requires an embedded stats engine and visual feedback for when these differences are statistically significant.

Other features may include in-dash experiment metadata (hypothesis, variant definition, effective dates, etc).

Market A/B testing tools typically come the latter feature set but they come with their own trade-offs.

The most flexible way is to join your experiment segmentation to the data in your warehouse.