Assumptions

1. I am going to assume that we are talking about the in-app notification panel on Facebook.com’s web and mobile apps
2. I’m also going assume there is no option to show both types of notifications
3. We’re considering only first-degree connection as a friend
4. We’re looking to develop a hypothetical algorithm that will match notification type with user.

Product
I have used Facebook Notifications before but I want to make sure I fully understand the product before I go on to discuss tradeoffs. FB Notifications is as old as Newsfeed. It is one of the primary touch points for people to stay updated about a variety of events on Facebook. It helps drive Facebook’s mission forward of bringing the world closer by keeping users engaged around events- their own, their friends’, communities and the world. Users can choose to view notifcations and take actions on them.

I’ll outline how I would like to approach this.

My approach:
I want to set a goal for the Facebook Notification in relation to Facebook’s Mission and the current state of notifications.
Next, I will identify hypothetical factors to consider for notification algorithm to show one type of notification to the user.
We can draw a few hypotheses for each type of notification under consideration, maybe run an experiment to validate the algorithm and discuss trade-offs.

Goal
As a goal for a mature product, I want to make sure people are not only engaged but are also coming back for more updates from their notification panel. (Retention). I can achieve this by aiming for
a) Relevancy of notifications with user
b) Consistency of notification experience across web and mobile
c) Timeliness of updates

Hypothetical Factors
The algorithm is going to rank and notify notifications to the user based on an exhaustive set of factors to achieve the goal of retaining them. However, for the sake of this discussion, I will limit it to a couple of important factors in no particular order.

1. Lifetime on platform (Newly signed up or Early adopter)
2. Platform usage (Power user, Seasonal, Dormant)
3. User Age (Financial signals such as 15 year old with no money to contribute/ 27 year old with disposable income)
4. Friendship Signal (Strong meaning overlapping interests and/or past interactions with the connection to Weak)
5. Feature Interaction Score (Number of times user interacted with the event / fundraiser feature * average session duration).
6. Location (Within the range of user’s location or not)
7. Time (Time event is scheduled to happen, Time fundraiser expires)
8. Timezone (User in same timezone as friend or not)
9. Geopolitical events with relation to user’s location (COVID surge in India vs. Juneteenth Celebration in USA)
10. Notification preferences (All, Highlights, Friend’s posts, Off)

1. User will be aware of the fundraiser/ event and will develop consideration to donate/ engage at some point.
2. If they do donate/engage, it will make the user strongly invested in the cause/event and thereby help the community come closer.

1. User might already be aware of the fundraiser/ event on other FB touchpoints such as Newsfeed, Stories, WhatsApp, Instagram, Messenger etc. thereby making user feel the notification is duplicate/ aggressive
2. User might have already donated/ attended event on platforms outside of FB thereby making user feel overwhelmed
3. If user is not into philantrophy or has personal financial struggles themselves, they might be inclined to ignore or worse turn off the notification
4. If the algorithm predicts a 50-50 probability for either, we would have set secondary criteria to select notification

Give me your thoughts on my answer to this Facebook execution PM interview question.

Clarification and Goals

By notification, do you mean the items that are displayed when you click the bell icon in the Facebook website and in a similar fashion on the Facebook app? Does this question incompass both the Facebook website and mobile apps? What about push notificaitons on mobile?

Let’s assume it incompasses all that.

Great. Can I assume that this is a mature product? Since notifications, events, and fund raisers have been in production for a while now.

Yes. You can assume this.

Great. Next I want to set up some baseline goals so we can work towards evaluating some trade offs.

• FB goal/mission: Keep users engage, bring the world closer together
• Facebook Events goal: Empower users to create events to bring family and friends together (Aligns well with FB mission of community)
• Fund Raiser goal: Raise funds and awareness for a partiuclar organization/event (Brings community together around a cause)

Users and customer journey

Next I want to go through the customer journies of the target personas to get an idea of what sort of pertaining metrics we can track in order to asses trade offs on a quantatative level.

Events
Create an event -> Invite friends to event -> Friend sees notifciation -> Taps notificaiotn -> Friend does 1 of four things
1. Nothing
2. Comments
3. Marks themselves as interested
4. Marks themselves as going

Fund Raiser
Create fund raiser -> Send out invites for fund raiser -> Friend recives notificaoint -> Friend does 1 of 3 things
1. Nothing
2. Comments
3. Donates

Now let’s list some pertaining metrics to each user

Events Metrics
– Number of users invited
– Number of users who viewed the event
– Number of users who marked themslves as Interests or attending (Converstion)
– Average % of users (out of all invited) who viewed an event
– Average % of users who converted

Funder Raiser Metrics
– Number of users invited to fund raiser
– Number of users who donate (Conversation)
– Total amount of funds raised
– Average number of funds raised (Across all fund raisers)
– Total number of funds raised (Across all fund raiser)

Evaulating Trade offs

Now that we have an understaning of the user, their customer journey, and some pertaining metrics, I think we are now in a good positions to evaluate, analyze, and make appropriate trade offs.

I think at the end of the day, as a Facebook PM in charge of notificaiton, I want to delivery relavant notification to my users so they are engage and convert.

With that said, I would develop a high-level hureistic/algorithm to determine the relvancy of a push notification. A higher relavancy would mean the user is more likely to convert.

Events Relevancy = Degree of clossness  + Location of event (closer is better) + How far out the date of the event (closer is better)

Fund Raiser Relavancy = Degree of clossness  + Cause correlation/relvancy with my likes/interest

*Degree of clossness = Determine by how often the user interacts/discuss/talks with the creator (of the event or fundraiser) throughout FB. e.g. you are more likely to repsonse to a close family member that you keep in touch vs someone from work that you never talk to or see.

Now simply, when deciding between a push notificaiton for a fund raiser or event, priroitize the one with the highest relalvancy. 

Selecting our key metrics

Now that we have some hueristics, this is what I would measure through A/B testing:

Event conversations vs Fund Raiser converations

In order to keep an eye on feature canoblizations, I would also look at the following as secondary metrics:

Fund Raiser
– Gross Funds raised,
– Average funds raised per fund raiser

Events
– % of invitees who viewed an event
– % of invitees who marked themselves as interested/attending.

Note: Since we are dealing with both web and mobile push notiification, it would be worthwild to track the conversation across platforms also.

Summerization

• We want to evaulate the trade off between deliverying a fund raiser notifications vs events notification.
• We took a look at what makes each of this notificaiton relevant to the user.
• Based on a hueristic (taking into consideration the event/fundraiser creator and the user recieving the notification), we will decide which push notificaiton to deliver.
• We will then meausre the converation and keep an eye on some high level success metrics for both Fund Raiser and Events.