What I understand from the question is we have to define goals for facebook notification feature and not actually list out the metrics.

The defined goals always need to be quantitative, that would tell us whether the feature that we released is doing well or not. If we are not able to achieve those quantitative goals, it’s time to think, if there’s a problem with the feature, or if we really need the feature at all.

Let’s list out types of notifications sent by Facebook.

1. Direct Message notifications
2. Like my page notification
3. Fb events invite notification
4. Birthday notification
5. Like, Tag and comment notification ( Photo, Post )

Let’s take the “Direct message notification” and try to define a goal for the same .

 

1. Direct Message notifications – users interact with messages in 3 ways ,
   • Sliding down the notification and reading the message
   • Clicking on the notification , reading the message and closing the app
   • Clicking on the notification , reading the message and replying to the message.

 

If we follow the HEART framework , where T stands for task success,

All 3 of these can be considered as Task Success for this particular feature.

 

Hence, In the case of “Direct message notifications” our goal is Task success.

 

Now , a quantitative and measurable goal would be that,

 

75% of the users should interact with “Direct Message notifications” in one of the above-mentioned ways.   

 

Clarification: I would like to understand this feature better. Are you referring to the bell icon on the Facebook page, which calls to attention important stories that the user would probably be interested in, along with key reminder for events such as a birthday of a friend or an event that the user has shown interest in attending through the Facebook Events feature? Answer: yes
Assuming core objectives of this feature are to drive user engagement and to align with Facebook’s core mission of bringing the world closer together. Is there any other objective that I should consider? Answer: No
KPIs/Metrics
Here are some key metrics that I would look at to measure the success/performance of the notifications feature:

1. CTR(Click Through Ratio) of notifications – How often does the user actually click on the notification to actually see the post.
2. CTR of story updates from friends vs forum activity vs post on a marketplace notifications. CTR actions on adverts can be interesting from a revenue perspective as well.
3. Ratio of actual action performed after opening posts (likes, comments, shares).
4. Additional connections (friendship requests) made after performing likes/comments on a shared post.
5. Ratio of remove notification action performed.
6. Ratio of action performed to turn off notification about a certain notification type(example: stop reminding me about birthdays).
7. DAU data based on changes to daily notification and measured through A/B tests.
8. User retention and churn on a weekly and monthly basis.

Answering any type of metrics question starts with asking clarification questions.

Clarification questions

1. Is there a specific goal that you would like me to focus on? (No)
2. For the sake of this question, can I focus on the desktop experience rather than on the mobile? (Yes)
3. In Facebook, there are notification badges on many components as Pages, Watch, Groups, Emails, and more. By saying notifications, are you referring to the notifications bell at the top and the section when clicking on it? (Yes)
4. Is it ok to focus on regular users instead of businesses or group admin? (Yes)

 

Structure

I’ll review the company’s mission and how the product assists in promoting it; Then, I’ll review the user journey, the different types of notification and define metrics to measure the success.

 

The Mission and the product

The mission statement of Facebook is to (1) make the world closer together (2) make tools for communities.

The main goal of the feature is to encourage engagement. Since there are so many areas and things going on on Facebook, the notifications section was created to help users not miss something that they might be interested in. There is a social agreement that people should be responsive, and this responsiveness assists in maintaining relationships and discussions.

 

The user journeys

The primary user journey is (1) The user notice that there is an unread notification (a red badge), (2) the user clicks on the bell icon, and it expands the dropdown with the latest notifications ordered by date (3) The user clicks on a notification and being redirected to the relevant pagesectionpost, or doesn’t click on the notification.

Additional scenarios:

(2) The user is not willing to see such kinds of notifications and adjusts the preferences.

(3) The user is willing to view all notifications on a full screen on navigates to the notifications page.

 

Notification types

1. Birthday reminder
2. Someone reacted (comment, reaction, or share) on a content you shared (post, story, image, or comment) or content in which you are tagged.
3. Updates from friends – someone posted, reacted, changed status
4. Tagging – someone is trying to tag you or already did.
5. Friend requests
6. Groups’ updates
7. Events updates
8. Marketplace updates

Note some of the notifications are on time, and some are aggregations of multiple updates. E.g., 40+ items were just added to your buy and sell groups; person 1, person 2, and 15 other people reacted to your post.

 

The Goal of Notifications

Share the right notification at the right time to increase engagement.
Clarifications:

1. The right notification – a notification that is relevant for the user, and we believe that she will be happy to know about it.
2. The right time – some notifications should be sent in real-time while others should be postponed or aggregated.
3. Increase engagement – there are two levels of engagement:
   1. Click on the notification a get to the relevant pagesectiompost.
   2. Engage once you get there – watch, comment, share, reaction, spent more time in that area.

 

Metrics

General Metrics

1. [Awareness] #/% of weekly users click on the notification bell.

 

Primary Metrics

1. [Usage] #/% of users click at least on a single notification during a session.
2. [Depth of usage] Average and Median clicks on notifications during a session.

 

Secondary Metrics

[Effectiveness] Track the effectiveness of each type of notification.

1. [Relevance] – Clicks/Views.
2. [How it promotes FB’s missions] – % Yield an engagement (message, comment, reaction, share, watching the video,…)
3. [We don’t want to spam] # of weekly notifications presentedgenerated per user – there is probably a general golden number. However, certain groups may need to have some other amount.

 

Counter Metrics

1. #/% of users without or with just a few notifications.
2. Complaints and adjustments in preferences (track it per notification type).

 

Concerns

1. How fake users are affecting the metrics.
2. Correlation between clicks on notifications and time spent on site (it can also negatively affect some users).

 

Summary

 

The goal of Facebooks’ Notification is to share the right notification at the right time to increase engagement. We will track the effectiveness of different types of notifications on the different levels of users’ engagement.