This is a hilarious story. Two weeks worth of work and 19 million in incremental revenue.
I have to thank Monica Chin and Calvin Colton. They were part of my Growth Team at the time and did all the heavy lifting.
- Category: Travel (Tours and Activities) & Ecommerce
- Location: Product page
- Opportunity: Over 10,000 products/SKUs are referenced on this product page. That is a lot of traffic.
- General Idea: Big opportunity, considering the amount of traffic. Let’s hammer away on this product page and get a better understanding about what our end users are thinking – What is our “aha moment” here?
- General Hypothesis: What can we change on the template (product page) to push more end users one step further down funnel? Can we assume, by pushing more visitors one step down funnel, that we can gain more revenue in tow?
I set the foundation. Our guiding philosophy was speed. This meant Monica and Calvin were tasked with increasing the number of a/b tests launched each month. If we run two a/b tests a month, let’s figure out how to run 4 a/b tests the following month. If we did four last month, how do we do eight the next month?
Our first four tests provided flat results. We did rather extreme redesigns, too. We also factored that into the equations for further testing. Meaning, “Hey, design is not the issue, something else is going on”.
Coffee and more thinking
Our working environment was interesting at the time. The office was very quiet – acute silence can be distracting. I dragged Monica and Calvin out of the office to encourage a little daydreaming and thought. They explained they might not hit their monthly target for the number of a/b tests in a month. I reassured them that they could hit their target with a simple button a/b test.
We examined the call to action and noticed it read “Add To Cart”. This was a seemingly appropriate CTA for an ecommerce scenario. However, our visitors were shopping for tours and activities, which isn’t implicit to having many SKUs in your basket. We also understood from our average order value that our end users were buying 1 item.
Now, I reflected back to what our Data Science team explained: “During high season our visitors are not aware that some tours and activities do and will sell out”
I told Calvin, let’s A/B test “Check Availability” vs “Add To Cart”.
Now, none of us were particularly thrilled about this a/b test. Button a/b testing is unoriginal but we had to hit our goal. Speed is the issue.
After about 2 weeks worth of a/b testing data, we discovered the little a/b test sent 50% more visitors down funnel. Yes 50%. We let the test run for another 2 weeks (1 month in total) and reviewed the results with our Data Science team. Well, turns out we also increased bookings by 6%. Not bad.
Curious to learn more about this a/b test? Read Part 2 here.