Meta (Facebook) wants to split its newsfeed into two. What metrics would you look at to validate this decision?

FB wants to test a new feature—split the News feed into two—Friends and Family Feed and Media Feed (clarified commercial/ads/other). 1. How would you validate this and with what metrics? 2. What are the downsides of your metrics? 3. Why did you choose “X “ market as a lower risk for launch?

  Meta
Add Your Answer
Answers (1)

Clarification Qs:

  • What does “split” mean here? – split the UI and if so how (through tabs?)
  • What counts as “media” – clarified already
  • What is prompting us to consider this change? – any negative feedback from the users/ decrease in certain Newsfeed metrics like engagement etc/ any particular business goal?
  • is this across or platforms eg desktop/mobile ?
  • Is this for all regions?

Mission of FB: Fb’s mission is to bring people closer together by enabling them to build communities.

Mission of Newsfeed:

To show users the stories/content that are the most relevant to them.

Overall goal of Newsfeed: Being a primary source of content, the goal of Newsfeed is to keep people connected with their communities by sharing content and interacting with the content – this ties back into the overall mission of FB.

With this the Primary goals of Newsfeed are:

  • To increase time spent on FB Newsfeed
  • To increase interactions of users

Decision making strategy:

I would conduct A/B experimentation to validate split vs no-split.

If we find out that split view has better results, then I will conduct further A/B experiments on the UI for the same- eg 2 tabs (2 separate pages) vs colored outline for different posts etc

Hypotheses:

Splitting the newsfeed into two parts would help users give a structured view where they can bifurcate between content from their family and friends (close circle) and ads/posts from businesses etc (interests) – thus increasing engagement with newsfeed.

Metrics that I would monitor to validate that I would look into are:

  • DAUs and MAUs for Facebook as a platform – since newsfeed is the primary source of content and holds a good amount of real estate – should increase – P0
  • Avg time spent on newsfeed per user per day – P0
  • No of sessions per user per day
  • No of interactions per user per day through newsfeed posts – P0
  • % time spent on Newsfeed / total time spent on Facebook – per session – per user -P1
  • Avg revenue per user per day from ads – P1
  • NPS metric for facebook
  • Time spent and no of interactions of UI 1 vs UI 2 – P2

Conclusion:

I will prioritise the bolded metrics above based on the primary goals for newsfeed which time spent on newsfeed and interactions of users along with overall retention of the Facebook platform.

If these metrics show increase for the split view vs non-split view, I would then also look into the

  • ARPU from ads metric as well to make sure it is not drastically affected negatively –> as it can severely affect the revenue from Facebook Ads.
  • % time spent on Newsfeed / total time spent on Facebook – per session – per user –> to check if Newsfeed is cannibalising user’s time and how is it affecting other FB features

Further consideration:

  • In the future, ML can be used to inform the UI changes based on user’s interests, behavior etc
  • ML can be used to better the recommendation algorithms within split views to show better ads, placement of ads etc:
    • Example: if user has spent a lot of time on friends and family view, the first time he switches to the media view, should the first page be an ad or a business post etc