User retention was the single most predictive measure of a customer’s likelihood to renew with Lattice, but only ⅓ of users regularly used its tools. I designed and built an new user onboarding flow that drove a small but statistically significant 2% lift in user activation in the first 4 weeks through a 50-50 split test. This met the criteria for a successful experiment, so I launched the onboarding flow to all users and prepared for V2.

Lattice is a wide-reaching HR platform with many tools, but one of the most common pieces of feedback is that users don’t know where to start. Typically, we relied on the HR team (our primary buyer and decision maker) to onboard their company, but it required a tremendous amount of in-person training from our implementation team in the hopes that the HR team could become the teachers for their company.

As Lattice’s first full user base A/B test, the new user onboarding experiment taught us a lot. First, it became clear how difficult running successful experiments in a B2B environment is. A B2C company might run experiments that target a fraction of their user base that could still include upwards of a million users and only need a week to reach statistical significance.

Since B2B companies onboard users much more slowly, Lattice’s experiment group was less than 10,000 users, so reaching statistical significance was a positive signal that we could continue experiments like this to improve the product.

Even so, a 2% lift was smaller than we’d hoped to achieve. It was clear that changing user behavior would be challenging, especially within the constraints from our EPD team, but it was a foundation that we’d continue to build on.

Everyone hopes that the first version of a new feature is perfect and we never have to revisit it. Of course, that's wishful thinking, especially for a zero-to-one feature. There are many things I would have liked to improve on the next version for onboarding, including:

1. Decide on one primary metric and action.
‍For this project, I worked with Data Science to determine that successful user activation meant the user completed at least one of three key actions, depending on the tools that their company prioritized. I assumed giving users more flexible choices would be better, but it likely created more confusion. In the next version, I would test a flow that prioritized one core action to make it extremely clear what the user should do vs. what the user could do.
   
2. Provide some basic explanation of the tools.
In an effort to keep the onboarding flow quick and to the point, I assumed the actions I was highlighting to new users would be familiar enough in the broader context. Of course, nothing is that simple. In the next version, I would test adding messaging around, for example, what Lattice 1:1s does and why using it is valuable from the get-go.
‍
3. Apply cleaner design with in-house code.
Being constrained by a third-party tool meant that I could only do so much to make the design clean and on-brand. In the next version, I’d spend more time working with other EPD team members to clean up the design more and, if resources allowed, creating the flow in-house so we could unlock a more creative and engaging UX.