Aha Moment

If you have not read part 1, you can do so here.

Let’s recap: After about 2 weeks worth of a/b testing data, we discovered the button a/b test sent 50% more visitors down the funnel. Yes, 50%. We let the test run for another 2 weeks (1 month in total) and reviewed the results with our Data Science team. We also increased bookings by 6%. Not bad.

The 6% increase in total bookings is the kicker. It was a massive lift in incremental revenue and it just made us scratch our heads a little.

What was it about the button test? Let’s take a detailed look.

Recipe A: Add To Cart
Recipe B: Check Availability

Recipe A, can be understood as an explicit hard sell. “Hey, just buy this thing,” with a flat emotional connotation.

Recipe B, can be understood as a soft sell, that resonates exclusivity. “Hey, you might not be able to get this thing” with an emotional connotation that can provoke a sense of urgency.

They I realized “OMG, sense of urgency is our a-ha moment”

What did we do next? I looked at the following page, and the CTA read “Proceed To Checkout”. I told Calvin to A/B test “Confirm Availability”. Calvin laughed and said “Are you serious?” I said “do it”.

To reiterate:
Recipe A: Proceed To Checkout
Recipe B: Confirm Availability

We waited 2 weeks for the results.

We discovered: 19% more visitors pushed to the next page. We increased revenue per visitor by another 5%. Not bad.

If you take a moment and think about this process, you will find most people do not flush out the “why” factor from the win of an a/b test. The “why” is purely important for the sake of repurposing in other aspects of your business. These were some fun a/b tests in isolated areas. Think of all the additional up funnel, cross channel/ platforms where future a/b test could be launched – knowing what our aha moment is. 

Brion Hickey