Facebook wants to split its News Feed into two. What metrics would you look at to validate this decision?

Add Your Answer
Answers (5)

Clarifying questions:

  1. Is this just for the FB app or this is for other FB app like Instagram as well?
  2. Will this test be on a specific device or across all the devices? (e.g. web, mobile, etc.)
  3. Will this test be done on a specific geography?
  4. Will this test be done for FB users with a certain follower count?

The core question here is the why. Why would we want to split the News Feed into two? A possible hypothesis is that Ads is taking up too much space in the news feed which means FB users are less engaged with the feed that they actually care about. People are on FB because they want to connect with their friends and family so interlacing Ads in Newsfeed is impacting engagement. It possibly is leading to a lot of time spent on FB without real engagement which means over a prolonged period of time the users will stop coming to FB.

If we go with that hypothesis then the metrics I would really look for in the test are core engagement metrics to see if there is an improvement in

Primary metrics I would look for is:

  1. No. of likes/comments: Do users engage more with content from family/friends since the content is now much more focused.
  2. Time spent on the page: Does the time spent on the feed increase with engagement
  3. Click on ads: With the new segregated view will it have an significant impact on the ad clicks

Secondary metrics I would look for is:

  1. Logins. Does it increase because users want to come back and engage with more relevant content
  2. $ revenue from the ad

If the engagement metrics are improving while ad clicks are going down for the time being then we may be ok with that if engagement is the goal here but then I would also see if the revenue is getting impacted because it’s possible that the quality of ads which are getting clicked on are much better due to improved engagement hence the revenue is not getting going south too much.

Clarifying question

Do we know what the split is? I am thinking about a tabbed based split, one for family and friends and one for media, including content from creators I follow and ads. Is that what you’re thinking too? (Let’s assume that the interviewer says yes, this is correct)

Okay, here is my structure:

I’d start with an initial guess about why we might be doing this

Then, I’d look at FB’s mission and how Newsfeed fits into that, and also the overall goal of Newsfeed itself

Then, I’d look at the short term, medium term and the long term effects on each stakeholder in this decision

Based on this, I’d propose an experiment, and also the metrics to validate this split

Let me know if this sounds good? (Let’s assume that the interviewer says yes).


Initial Guess: My initial guess about why we might be doing this is because making such a split would do two things. One, as a user, when I am on the family+friends tab, I have a higher sense of intimacy. As a result, I’d be more open to engaging with people I see in this tab. Second, when I am in the media tab, I am in the mode of consuming entertainment, similar to when I watch TV. This would enable me to tune out everything else and only consume entertainment. So just to recap, this split would give me a higher sense of intimacy, and would enable me to be entertained more effectively. Is there anything I missed here? (Let’s say that the interviewer says no)

Mission: FB’s mission is to connect people and to give them the tools to build better communities. Newsfeed makes sense in this mission because it serves curated content to the users which they are more likely to engage with, which in turn helps them build community. In short, Newsfeed increases consumption of content, increases engagement and builds communities

Stakeholders + Effects:

Users: Short Term: increased feeling of intimacy, more focused entertainment, decreased ease of switching between the two

Medium Term: Increased intimacy leads to more connections and more intimate communities

Long Term: More and closer knit communities

Creators (Video creators + article writers): Short Term: Decreased views on their content, but they have alternatives like Medium and Youtube and FB is their secondary channel anyway. Decreased engagement and feedback. But again, they have primary alternatives

Medium Term: less than optimal views and feedback equals less than optimal experience. So fewer creators

Long Term: possibly, creators leave completely

I’ll take a pause here to see if you have any inputs here?

Advertisers: Short Term: Fewer ad views, fewer ad clicks

Medium Term: Grabbing less public attention through FB, increase in alternatives for FB advertising

Long Term: Advertisers leave FB completely, for alternatives

Facebook itself: Short Term: Increased engagement amongst friends and families. Decreased engagement with content creators and ads. Increase in time spent, because of the feeling of intimacy amongst friends and family.

Medium Term: Increased monthly retention because of increased engagement amongst friends and family, decreased ad monetization

Long Term: More connections, bigger and more communities because of increased meaningful engagement, decreased monetization through ads.

Just to recap where we are, <summarize what we’ve done so far>

Proposing a way to validate this split:

Having looked at these effects, I would imagine that Facebook’s main focus is on the users and its mission of connecting them. This feature aligns with that mission. So I am positively inclined toward shipping this feature.

However, as a business, it also needs to survive. So it needs to either mitigate the effect of reduced views and engagement with media, OR, look at alternative sources of monetizing. My inclination is to go for the former, because another unique selling point for FB is that it’s free for the users. Hence, here is what I propose, as a trial for a 1% user base containing a sample representative of our overall user base:

Split the newsfeed into two tabs: the first, default tab for the media, so that when the users come to FB, they see the media content first. This might attract attention and tempt them to view and engage with more media content. The second tab for friends and family. My hypothesis here is that this tab, and the feeling of intimacy it’d offer would become the main attraction for them in FB and they’d be willing to do a small additional task of clicking on this tab. I would launch this feature in a place where bandwidth issues are almost non-existent, like USA, Japan or South Korea

The reason I’ve chosen the US, Japan or S.Korea is because we want to mitigate the possible loss of views on media content. Any bandwidth issues or buffering pauses would only exacerbate this loss. Hence it’s important to first try this out in places where the internet is decently fast

To validate this split, I’d divide my metrics again into two parts, metrics that are proxy for user value, and business a. User Value: # of users adding at least 3 meaningful comments per week on the family+friends tab; # of users viewing at least 10 media posts per week. These two would serve as leading indicators for retention, so it makes sense to optimize for these two.

A major tradeoff that I’d look at would be how these metrics can be gamed. For example, just increasing the size or color of the ‘Comment’ option could attract more comments, but that won’t tell us anything about future retention. For that, I’d additionally track the weekly retention on the both the friends+family tab and the Media tab.

b. Business: distance scrolled in the media tab per person. Optimizing for this would make it great for the advertisers and the creators and hopefully prevent them from leaving, long term.

One tradeoff I’d look at is that this metric can be misleading. An advertiser doesn’t just want a ‘view’ from the user, it wants attention. For this, we need to also track the amount of time spent watching each media post

Just to recap, <summarize the answer>

Clarifying Questions

  • What do we mean by “splitting” the feed here? <split into 2: Friends and family and Media>
  • Is “media” content posted by pages vs. individuals? <Yes>
  • How did we come about considering this modification? <Want to see how we can segment our feeds>
  • How will this split feed function? <it will be in two separate tabs>

Facebook Mission:

Facebook’s mission is to create and connect communities around the world. Newsfeed plays a key role in this mission as it serves up the more relevant community updates to an individual.

Newsfeed Product Mission:

More specifically, Newsfeed provides a centralized place for user to stay up-to-date in their communities.

In this decision to split this newsfeed, we would likely need to re-consider our core mission.


Let’s take a moment to consider the users that will be impacted most by this change:

  • Content Creators:
    • Pro
      • In a more targeted feed, you’re capturing individuals who are are seeking out your particular content
      • Opportunity to
    • Con
      • Need to refine content strategy to capture this new dynamic
      • Uncertain about the efficacy of the two feed strategy
  • Individuals:
    • Pro
      • Can actively choose what content is most important
    • Con
      • Complicates a simple concept (mental load of deciding which feed
  • Advertisers:
    • Pro
      • Potential for better targeting based off of content
    • Con
      • Potential to dilute my current engaged user base

The theme you can see with the pros and cons are mirror ideas of one another– the potential for a big upside or downside depending on the implementation.

A/B Test

Assumptions: Users will spend more time cumulatively, or in either of the respective newsfeeds.

This test will be focused on whether or not splitting feeds based off of the suggested divisions will generate more time and engagement on the platform. This will not determine whether or not the subject matter of the split feeds (family & friends vs. media) is the optimal choice.

Control group: Access current single, newsfeed

Test group: Access to a split feed between friends & family and media

Specific to the test group:

From a more operational perspective, I would want to take a look at the following metrics to see from a more tactical and functional perspective

  • # of users who viewed both feeds
  • % of inventory viewed / % inventory available
    • Concern here specifically that we don’t have enough content in the friends & family feed

Metrics to compare between groups:

  • avg. cumulative time spent in newsfeed
  • avg. # of impressions / piece of content
  • avg. engagement [like / comment / share / react]/ content
  • avg. session time
  • revenue / 1k impressions

I would compare the metrics across these test groups to see which option nets out the the most engagement, specifically:

  1. avg. engagement / content followed
  2. avg. cumulative time spent in newsfeed(s)

Broader FB Consideration

If the split test nets out on top, I want to take a look at the implications of executing this change. A few things that come to mind include increased competition for feed-like products such as Groups and Facebook Watch.

Ideally, this split view will serve as a better funnel into these respective products; however, if engagement is impacted on these products because of the split view, I want to see how severe the impact would be and whether or not, from a strategic perspective, this is an acceptable decline.


Give tools for people to build communities and bring the world closer together through Meaningful interactions

News feed:

Surface meaningful posts to people to interact with to bring them closer together

(what is meaningful? It’s different for every person) Assumption here: close circle interactions are more meaningful for the majority of facebook users.

Stakeholders involved:

  • Post creators
    • Commercial/Ads
    • User’s network
    • User
  • Post Consumers
    • User
    • User’s network
  • Other teams & their goals
    • Groups’ engagement
    • Monetisation team etc.
  • Facebook itself

Product Change Benefits:

  1. For Users and their network: Drive more users to have close network interactions? > Meaningful engagement (Short-Term)
    1. Time Spent on friends’ feed/user/week (Vol)
    2. high value (shares, comments, RSPVs) engagement over total engagement /user / week (Depth)
  2. Positive Brand perception value for FB for everyday people -> Long-Term
    1. % increase in DAU/MAU (stickiness; long-term)
  3. For advertisers: Increase in engagement from loyal customers -> ST
    1. Retention: % of repeat customers (click-through again within a week)

Product Change Costs:

  1. Decrease engagement to Ads post (volume of ads) -> ST Ads revenue
    1. Total Impressions
    2. # / % of users visits the commercial news feed per week
    3. Total # Click-throughs on ads
  2. Negative FB Brand value for advertisers (monetisation capability) -> LT Ads revenue
    1. % decrease in ad spend on FB
    2. # of posts on FB
    3. $ revenue monetised per user

Common currency/Goal: There isn’t an obvious one in this case.

We can perhaps make a decision on a long-term Vision: What kind of company or product ultimately do we want to be in 10 years time? -> meaningful engagement metrics


A/B Test

Hypothesis: By splitting newsfeed, we can increase the overall high-quality time spent on network feed by minutes (if a user has carried out at least 3 shares or comments or RSVPs within a session, we call that a quality session, we add the overall session time up)

Long-term: % increase in DAU/MAU

Guardrail: While we want to optimize for the goal above, we need to measure “for every $ of Ad revenue sacrifice, what’s % of high-quality time spent on network feed we can increase?”

Test details:

Users: 1% globally; or pick a few smaller markets to A/B test this

Optimising the test groups:

  1. Primacy effect (new users vs. repeat users)
  2. Users who engaged highly with Ads in the past -> exclude them (“meaningful interaction” is different for everyone)
  3. Different demographics, different devices (perhaps network feed consumed more on Apps than desktop)

Optimising the Product UI:

  1. Tab vs. no tab?
  2. Add shortcuts between the 2 feeds?
  3. Defaulting the newsfeed based on the user’s usage pattern?

Other Considerations:

  1. Impact on other teams?
    1. Surfacing Business page’s update in Commercial feedback may drive their engagement metrics down
  2. The integrity of extra engagements generated from the network feed

Clarification Qs:

  • What does “split” mean here? – split the UI and if so how (through tabs?)
  • What counts as “media” – clarified already
  • What is prompting us to consider this change? – any negative feedback from the users/ decrease in certain Newsfeed metrics like engagement etc/ any particular business goal?
  • is this across or platforms eg desktop/mobile ?
  • Is this for all regions?

Mission of FB: Fb’s mission is to bring people closer together by enabling them to build communities.

Mission of Newsfeed:

To show users the stories/content that are the most relevant to them.

Overall goal of Newsfeed: Being a primary source of content, the goal of Newsfeed is to keep people connected with their communities by sharing content and interacting with the content – this ties back into the overall mission of FB.

With this the Primary goals of Newsfeed are:

  1. To increase time spent on FB Newsfeed
  2. To increase interactions of users

Decision making strategy:

I would conduct A/B experimentation to validate split vs no-split.

If we find out that split view has better results, then I will conduct further A/B experiments on the UI for the same- eg 2 tabs (2 separate pages) vs colored outline for different posts etc


  1. Splitting the newsfeed into two parts would help users give a structured view where they can bifurcate between content from their family and friends (close circle) and ads/posts from businesses etc (interests) – thus increasing engagement with newsfeed.

Metrics that I would monitor to validate that I would look into are:

  1. DAUs and MAUs for Facebook as a platform – since newsfeed is the primary source of content and holds a good amount of real estate – should increase – P0
  2. Avg time spent on newsfeed per user per day – P0
  3. No of sessions per user per day
  4. No of interactions per user per day through newsfeed posts – P0
  5. % time spent on Newsfeed / total time spent on Facebook – per session – per user -P1
  6. Avg revenue per user per day from ads – P1
  7. NPS metric for facebook
  8. Time spent and no of interactions of UI 1 vs UI 2 – P2


I will prioritise the bolded metrics above based on the primary goals for newsfeed which time spent on newsfeed and interactions of users along with overall retention of the Facebook platform.

If these metrics show increase for the split view vs non-split view, I would then also look into the

  • ARPU from ads metric as well to make sure it is not drastically affected negatively –> as it can severely affect the revenue from Facebook Ads.
  • % time spent on Newsfeed / total time spent on Facebook – per session – per user –> to check if Newsfeed is cannibalising user’s time and how is it affecting other FB features

Further consideration:

  • In the future, ML can be used to inform the UI changes based on user’s interests, behavior etc
  • ML can be used to better the recommendation algorithms within split views to show better ads, placement of ads etc:
    • Example: if user has spent a lot of time on friends and family view, the first time he switches to the media view, should the first page be an ad or a business post etc