How do you measure the success of Airbnb online experiences?
Google
Airbnb- Natasha Martin
Context
Great, first off let’s just spend a moment getting on the same page in terms of what Airbnb Online Experiences is and more importantly why users use it.
During COVID, domestic and international travel slowed to a halt and in parallel Airbnb bookings tanked as well. As a response to his Airbnb launched Online Experiences which are digital experiences users can book on the platform. For example, a Tokyo based Sushi chef could host an online sushi cooking lesson that users could sign up for. Hosts are able to continue to share their culture and generate revenue while travelers are now still able to experience new cultural experiences while being relegated to the confines of their home.
Before we dive into the nitty gritty, let’s first take a step back and make sure we understand the context surrounding the bigger picture. Airbnb’s mission statement is to create a world where anyone can belong. Online Experiences helps further this mission statement by both allowing people to experience the world when there are travel restrictions in place and beyond that by enabling those who can’t afford travel to still experience other cultures. For example, not e veryone can afford a $3000 trip to Toyko but a $50 online sushi lesson is going to be a lot more accessible.
Users
We touched upon it above but it’s worth calling out that there are two distinct user groups here. The hosts who are creating and putting on the Online Events and the “Online Travelers” who are attending these events. Both are critical to the success of Airbnb Online Events.
Metrics
Success and its definition varies from product to product and a big component of that is which stage of the product lifecycle that product is at.
For a recently launched product, like Airbnb Online Experiences, I would first want to address whether or not we’ve found product-market fit. Are we able to effectively address the needs of our users? More specifically, are we able to help Hosts make more through putting on digital experiences and are Online Travelers able to enjoyably experience other parts of the world from the comfort of their home.
The best way to measure product-market fit is through a combination of engagement and retention. If users are meaningfully engaging with the product then we’re adding value to them. If they come back and engage again then we’ve added enough value that they think it’s worth coming back to.
We could measure success and our product-market fit for Airbnb Online Experiences by examining the following metrics:
Engagement:
- Total number of event attendees
- Average number of attendees per event
- Total Number of Events
- Average number of events per Host
- Average ticket price
Retention:
- % of hosts who hosted 2 events last month who also hosted 2 events this month
- % of online travelers who attended an event last month who also attended and event this month
- Average event review rating
Dowsides
While I’m confident in the ability of our above proposed metrics to measure the success of Airbnb Online Experiences, no metric or set of metrics is perfect so I think it’s worth spending a few moments talking about some potential downsides or gaps.
- While intentional, all of our above metrics are fairly high level. They don’t give us much granularity in terms of how different different segments may behave differently. For example, maybe cooking events do really well or people with kids don’t enjoy Online Experiences.
- While we’re focused on the success of Online Experiences we want to make sure we aren’t negatively impacting other parts of the Airbnb platform For example, if people are enjoying the Online Experiences so much they may not be as compelled to book actual travel and lodging through the platform.
Summary
As Airbnb Online Events is a recently launched product we are going to measure our success by whether or not we’ve achieved product-market fit as determined by the above metrics monitoring engagement and retention amongst both Hosts and Travelers.