Rider experience for Uber Airport rides is down in the last 6 months. What would you do?
Uber- Rohit Kumar
Context
Before we dive into figuring out what’s going on, let’s just make sure we understand the bigger context. Uber itself is a ride sharing company where users via a mobile app request rides from point A to point B which are then accepted and fullfilled by drivers on the platform with Uber getting a cut. As a company Uber’s mission is to make transportation as reliable as running water and Uber in the context of the airport helps provide transportion itself to the airport and then it also secondarily makes flying itself more accessible by doing that.
Clarification
Now that we understand the context, let’s make sure we understand the question properly. I have a few questions / points in that regard:
- When we say rider experience is down, how exactly are we measuring rider experience? What metric or set of metrics is being used? (Probably end of ride survey responses)
- What is the maginute of the drop we are witnessing? 2% vs 20% are completely different. We should also confirm that this drop is statistically significant.
- Going to make the assumption that we are confident in the accuracy of our data. Nothing is more heart wrenching than finding a bug in the reporting or collection of data you’ve been using to guide your decision making.
Honing in on the issue
Right now we have a general observation, let’s see if we can try to hone in on the issue or isolate it further by asking some questions:
- We said it was down over the last 6 months. Did this gradually decline continuously over the last 6 months or was there a sharp drop?
- Is this uniform across geographies, or are there particular cities/countries where the drop is more or less prevalent? (Probably want to control for the number of airports when looking at this)
- Is the drop uniform across various user demographics and user segments? For example, Uber Pool vs Uber Black, Young Riders vs Old Riders, New Sign Ups vs Power Users? To the airport or from the airport?
- Any disparity amongst technologies or platforms? iOS vs Android? App versions?
Potential Causes
With answers to the above questions we can hone in on the issue further and brainstorm some potential causes tailored to what’s happening. We don’t have the answers so let’s just brainstorm some generic potential causes and more importantly how we would validate them.
Internal (Things we did to ourselves)
- Product Changes
- Could we have made the UI clunkier or harder to understand for Airport specific use cases? For example, maybe we made it harder for users to designate which terminal they are flying out of and need to be dropped off at. — We would examine the product release notes around the time the drop started happening and look for anything related to airports. If possible look at A/B test results for UX data related to any suspicious release.
- Did we introduce any bugs in recent releases? Maybe a bug is causing our routing to function poorly in situations with many overlapping roads which is common in airports — Ask QA/Dev about any known bugs which might be relevant and examine pieces of functionality touched by recent releases.
- Have we released any new marketing campaigns that could be misleading users or giving them unrealistic expectations? For example, if we are using a tag line like “Book an Uber, make your flight with time to spare!” then I could see users being upset with us if they miss their flight whereas in reality that may be out of our control.
- As of the last 6 months, are we surge pricing users in airports more than previously? If riders expect a 30 minute ride to cost X dollars, but we are charing them 2X they are not going to be happy about it. — I would look at the percentage of airport rides that were surge pricing rides over time.
External (External forces at play)
- Could there have been new laws or regulations passed in regards to how ride shares are facilitated within the confines of an airport? For example, maybe now ride share drivers are only allowed up to 5 minutes of idle time before they must drive off which might result in users missing their rides. — I would use google to search for recent regulation changes.
- Could airport ride demand gradually outpaced ride supply leading to longer wait times and worse user experiences? — I would examine the average wait time of users near airports.
Summary
We’ve observed a drop in rider experience around airports in the last 6 months. First we want to understand what metrics we are using to describe this drop and then we can try to isolate this futher by looking at a few different elements such as: rate of decline, geography, platform type, and user segments. When we’re able to isolate what we’re observing further we’ll have a better sense of just what the cause is but we’ve gone head and proposed several pontential causes and ways to either validate or invalidate them.