Clarifying questions

I want to confirm that the reactions mean like, love, etc. for post right? Did we double-check the data?

External factors

• Is it a WOW metrics, MOM, or DOD metrics
• I am assuming it’s a MOM change
• Is this related to any specific geographic region?
• Do we have any big events going on around the world
• Did we have any bad PR and press release that could have caused this downtrend in comments
• Is this trend may be related to specific user ( Genz, Mills, or baby boomers )
• Is this trend specific to any platform ( Desktop or mobile ?)
• Different OS?
• Different versions?
• Seasonality

Customer journey

• Were there any new feature release that we know from the comments or the side of the reaction?
• Can also assume that the Facebook reactions is a feature that was not rolled out recently?
• Did we make any changes on where the comment field appears for posts
• Or did we actually may be made changes on how the reactions appear ( making it big or small etc )

Internal factors

• Are there any bugs in the comments feature
• Are the comments feature down are people being able to comment?

Hypothesis

• If changes to the reactions feature like increasing the icon size making it more visible make the improvement?
  • The trend MoM after and before the change
• I would want to understand if the placement of the comment box has changed or has the size changed
• • The trend MoM after and before the change
• Understanding if people were using reactions to express themselves compared to commenting on posts
  • Want to understand how trends are changing as people understand using the reactions more?
• Are people not using reactions because comments were down?
  • Whether the comments are working for people or not?

Answer: Clarifying Questions:

1. Since when has this changed? Any correlation with seasonality?
2. Correlation with platform (web/app traffic)/geography/user category (teen-age vs retired etc.)
3. Sudden or gradual change?
4. Did we make any release just before this change
5. External Factor – update to privacy policy, regulations; competitor product

Approach: Map customer journey

1. User logs on FB
2. Sees a post on Newsfeed/Notification
3. Goes thru the post
4. Gives feedback: like, comment, skips and this would be across products like reels, posts

Metrics to be looked at: Gives feedback: like, comment, skips. Hypothesis for the above is that the change has to do either wrt a post type or user type

1. % like vs comment in reels vs post : trend for a month -> in case data suggests issue, then will get the flow QAed for RCA and fix
2. % like vs comment basis user age type (demography) : trend for a month -> in case data suggests issue, then will work with product marketing or business team for user retargeting and engagement based efforts

Also important to note that business value if still is increasing with this change in trend then this may be construed as a positive change.

Problem: Reactions are up by 10% and comments are down by 20%

1. Purpose of Reaction (Goal), value Reaction and comments are the key ways for meta users to engage on the platform. Value: Viewers: get value when they find content which resonates with them, and they decide to engage in the form of reaction or comment. Creators: get value when the content they have posted gets engagement from the viewers in the form of reaction or comments.
2. Hypothesis Reaction: takes less time, so easy for users to give, hence the number is up. Comment: takes more time, so users prefer to avoid them, hence the number is down.

• Metrics I will use to test “More interactions across FB”: Avg # of posts with >1 reaction per week, Avg # of posts with >1 comments per week
• Metrics I will use to test “More posts”: Avg # of posts per week, Avg # of posts with >1 reaction per week / Avg # of posts per week
• Metrics I will use to test “More Users”: Daily Active Users, Avg time spent on the platform (*Cannibalization)

3. Key Success Metrics I will use to test my hypothesis Avg # of post per week Weekly Active users Avg time spent by users per week
4. Data-driven Decision => ship, no ship, or retest a) Ship: Avg # of content per week (up) Weekly Active users (up) Avg time spent by users per week (up/down) (If it is not down significantly, I will be ok with shipping) Reason: Metrics are up proving the hypothesis correct.

b) No Ship: Avg # of content per week (Down) Weekly Active users (up/down) Avg time spent by users per week (up/down) Reason: If the Avg content is going down WoW, it means in the long run viewers will have less content to engage with and they will prefer other platforms where they can find more content.

c) Retest: Avg # of content per week (Up) Weekly Active users (down) Avg time spent by users per week (up/down) Reason: WAU, while important, could have been down due to some external factors (holidays) or internal factors (bug). So, I would retest the A/B testing for a longer duration so that other factors can be smoothen out.