Inbox

Product Lead — 2017-2019

The discovery, design, & development of the award-winning game, Inbox.

Driving Question

What could a modern business training experience look + feel like?

The scoop

Roles — Product Lead (Sr. Product Manager). Later, Director of Product Design & Development

Team — Initially, 2 full-stack engineers and myself. Later, a team of 15 designers, developers, analysts, testers, and Product Managers.

Goals — Expand our reach into corporate learning & development with a high-fidelity management training game. Build a new business line and validate growth.

Results — 4x EdTech Digest award-winner (2020, 2021, 2022) for Best Corporate Training Solution, Best Professional Skills Solution, Best Hiring Tool, and the Workforce Trendsetter Award (over Salesforce and IBM Watson).

50+ custom training simulations. 10,000+ users at Fortune 500 clients and top universities worldwide.

Skills — Product Leadership, Product Discovery, Conceptual Design, UX Design + Research, Information Architecture, Go-To-Market Strategy, Market Development

Discovering the Opportunity…

While studying for my MBA, I underwent a management in-basket training exercise. The content was great — a series of emails, interwoven and one-off, that I had to decide how to handle — but it was wildly outdated! Hundreds of us were asked to write out our responses as memos. It felt too easy...

From my time building new simulations — both with internal stakeholders and and custom designs with external consultants — we knew the impact of experiential learning. People learn best by doing. How might we bring this to the executive education experience?

So we focused our effort on finding where modernization would fall short. Why hadn't this idea caught on?

The qualitative research I led put me at tables with Chief Training Officers and Heads of Learning & Development, as they shared how their people learned, failed, and how badly they needed experiential learning. We formed large-scale surveys of executives at corporate universities and business textbook authors, and heard more of the same.

The Design Playbook I put into practice to lead our exploration had worked. Of course it did! We adopted these best practices from IDEO and Stanford’s d.school, and iterated our methods over many years. We closed out with a prototype concept of an email interface that acted as your game dashboard. Below you can see some images of that process and the living document that guided our work.

testing the idea

So, we modernized the in-basket assessment, creating a refined form with built-in feedback with a brilliant organizational behavior expert.

This occupied every night of my summer of 2017. Alongside two full-stack engineers, I led the product design, product management, and translated our expert's scenarios to help author our first Inbox game.

We built the initial prototype over 2.5 months in Node.JS with a Bootstrap 3 UI library. Throughout this period, we shared early prototypes and workflows with our initial research group.

One minor surprise came at this point — the game wasn’t enough for the Chief Training Officers to sign off on. They wanted to 1) know how their people compared to similar groups, and 2) ensure their people acted on the feedback they were given. So we went back to the well…

Measuring Success

In the fall, we launched our pilot edition of Inbox, titled Inbox: General Management. We were off to the races.

This pilot version walked a cohort of 20 L&D teams through a 45-minute simulation, gave hands-on feedback following the experience, and put learners through an Individual Development Plan that worked back through areas they needed to improve upon. In total, 400 Masters-level corporate employees and consultants built out our initial database.

This ended up being a critical design choice, as Chief Training Officers could view their group’s performance in a multitude of ways. Because scoring was on a percentile scale, it wasn’t just a matter of employees being ‘wrong’ or ‘right’. This enabled L&D teams to use Inbox as an performance indicator alongside similar employees inside and outside their organization — how do you rank alongside your peers? We later used this to fit into the Annual Review process.

Because we wanted to validate the opportunity in the corporate L&D market, we focused our key metrics on Revenue Growth and NPS.

Within our first year, we logged more than 10,000 users, with 40% coming from the corporate L&D market. Because Capsim is privately owned, I will not disclose the financials here, but our test was wildly successful. NPS for our first test came in at 42, which didn’t thrill me, personally, but beat the benchmark of corporate training materials (which at the time was an NPS of 20).

Side Study - Measuring Effectiveness IRL

A game can be great, but if it doesn’t make a difference in real life, then it falls short.

Inbox was designed to recreate realistic work environments. Learners experience critical incidents related to their world and then must respond accordingly. These scenarios were rigorously tested to ensure their impact.

As experts in experiential learning, we used our pilot study to test the effectiveness and compare to on-the-job performance, as measured by 360 reviews in their current role.

Full details of our study live in a Technical Synopsis, published by Erich Dierdorff, PhD. You can read more here.

Suffice to say, Inbox effectively measures job performance. We were successful.

Scaling

Once we had brought this to market, we realized the need to evolve our prototype designs. Due to the success of our initial tests, I was given rein to bring designers into the project.

One of the first steps was to standardize all components within our Inbox tool through a design system. Design systems have of course become in vogue since then, but at the time we had made huge advances to our design-developer handoff process.

Reflections

While the memories have faded a touch, I remember being fully absorbed building Inbox. No matter what it took to make this a reality, we did it.

This meant many late nights, a newfound role as the internal and external voice of Inbox, and a crash-course in big data analysis.

So much came with inventing Inbox — namely our discovery of the expert-creator gap, which you can read about in Case 2 — Enabling Experts to Author Inboxes.