This was a really fun Facebook technical question to answer. Its very easy to make things more complicated 😛
I will focus on three specific areas to answer this question:
- Framing:
- What problem are we solving and how it relates to the company mission
- Users
- Goals:
- Goals of this (data) product
- Establish a criteria for success
- Solutions:
- Review of existing product functionality
- Propsed Metrics
Initial Problem
Many sellers don’t mark items as sold on FB Marketplace, what existing functionality and metrics could be used to determine whether an item has likely sold?
I would first confirm that – based on the phrasing – the interviewer wasn’t restricting the answer to use existing metrics. I will assume for this the answer is no – existing functionality, and any metrics I can come up with based on those functionalities and known data.
Linking to the Company mission
The company mission is worth keeping in mind here: enable people to build communities and bring people together. While a marketplace is not expected to trigger new friendships, a busines and less formalised trading between people are a postive reflection of a community with trust and safety. Whether physical or online/virtual, it is vital there is trust between buyers and sellers and also in the quality of the product being traded. The transaction must also be trusted as safe, or with a mechanism to reclaim ones money back in the event of fraud.
Much of that trust is dervied from the platform, who is expected to provide tools and services in the form of buyer/seller ratings, supporting issues with bad actors etc. In the real world, when we shop in a market we assume a local council or market organiser have vetted the stall owner in some way – in other words, our perception of a maket and its sellers are interconnected. If we have a bad experience with one seller, we may lose trust that other bad actors are not being weeded out.
People are able to share posts about a new marketplace item into FB Groups, these communities need to trust what they see are as described. Relating this to the problem at hand – people need to trust that the items they see are available and not already sold.
We will take the overall mission of the FB Marketplace team as: enabling people and businesses to safely and efficiently buy and sell goods to other FB users. FB Marketplace does not facilitate delivery or the exchange of money.
Users
The user we are focusing on is the seller since the question states that many sellers don’t mark items as sold. In other words, it’ not every seller. Sellers have to manually mark an item as sold, so why might some users not do it?
– The seller is not aware of the feature to mark items as sold
– The seller forgot or is not in the habit of doing it
– The seller sees no advantage (or no motivation to avoid a potential disadvantage) by not doing it
– The seller is deliberately not doing it
Some of these overlap with possible user categorisation relating to the level/type of marketplace experience a user has:
– new/first time seller
– infrequent seller
– frequent seller
– formalised business (which can then also be either a new, infrequent, or frequent seller)
– a criminal or fraudster
Plus of course, sellers are rated on the marketplace! However, ratings are not specific to an item, just to the seller
Additionally, Facebook has data on the friend community of each seller on Facebook itself including Messenger, and potentially on Instagram and Whatsapp also. The account itself may hold clues in terms of online behaviour with the platform(s) and with friends.
To summarise, the existing functionalities relating to users we can utilise and analyse data about are:
1. Seller ratings / feedback / reports from other users
2. Seller behaviour on FB marketplace
3. Seller experience and business status on FB marketplace
4. Seller’s personal FB / Insta / Whatsapp networks and behaviour
Goal
Our goal is to reliably identify items which are likely to have been sold but have not actually been marked as sold by the seller
Success:
a high degree of accuracy when identifying items are being likely sold, but not marked as sold by the seller
Existing functionalities:
1. Buyers rating sellers: After trading with a seller, a buyer will rate them. True and fair buyers would not rate a seller if they have not purchased from them. So, assuming we have high confidence that a buyer has actually purchased an item we can utilise this in our metrics.
2. Marketplace seller ratings: Based on when a seller recieves a rating
3. Behavioural data: Segmentation of sellers based on their behaviour, status and potentially their personal network.
4. Messages: NLP analysis of buyers’ messages to sellers asking if items are still available, which have gone unanswered
5. Data analysis and machine learning are applied to create a confidence score for all items not marked as sold. Utilising the experitse of data science we could apply ML techniques to analyse items. However, depending on the ML technique(s) applied it may require good training data. FB has scale, but the training data would need to include items that are known to be sold and known to be unsold. Given that we are trying to identify the latter, some ML techniques may not be appicable. I would work with data science to see what is possible.
6. People reporting items to Facebook directly as listed incorrectly: This is the easiest and most convenient, but works on the assumption that people will report items as not properly marked. There is no mechanism for buyers to confirm they have recieved the item (which there is on eBay for example), which would also help to infer an item was sold.
Metrics:
Based on the above, possible metrics could be:
– confidence score for sold items: could be derived by data science team or by a formula using multiple metrics below.. assuming there is a confidence score we could simply have:
– count of items likely sold (filtered by those where confidence score is high; assuming a simple categorisation of Low, Med, High)
Metrics (assuming the above confidence score is NOT available):
The metric names will likely be verbose to make them clear, I would find shorter and simpler names with more time…
– Inferred purchase: Using data that a buyer has rated a seller to infer an item was purchased. The difficulty here is identifying which item a person has actually purchased as the transaction and shipping are not facilitated by FB. However, we can utilise data about items a person viewed, liked, saved, commented on or asked the seller a question about to infer which item it was.
– Unanswered item status requests: the number of times buyers ask if items are still available which were not answered or not answered in the affirmative.
– Item time on marketplace or “time since posted”: how long an item has been on the market (this assumes older items are more likely to be sold, which may not be true for some items which are not at all popular and where the seller wants to hold out for a sale)
– Sellers not likely to mark items as sold: (based on the behavioural analysis mentioned earlier). This assumes a confidence interval can be created from the analysis of behavioural data – this would enable a ‘grade’ or index to be created. Based on the grading, this metric only counts the sellers above a certain threshold.
– Fraud flag: true/false to indicate that a flag marker is placed on the account due to bad ratings, comments, or reports from other marketplace users.
Using this data, I can see the potential to create a mechanism to send notifications to sellers to simply ask them if an item has sold. Ideally, that seller could answer Y/N directly in the FB notification to reduce friction and remove the need for an extended user journey. If users have multiple suspected unsold items, then targetted training messages could help e.g. an email that explains the situation with their items and shows them how to mark them all as sold.
Meta