You are the PM of Messenger, you noticed that its DAU has gone down significantly. How would you go about looking for the root cause?

  Meta
Add Your Answer
Answers (2)

First, I would need to clarify certain points :

  • DAU = Daily active users. How is that defined ? Does the user need to take a specific action or does he just need to open the app ? For desktop, if he goes on FB, does he count as a DAU messenger ?
  • Which device was the decline observed ? Let’s assume it is Mobile
  • How much is significantly ? Let’s assume it is around 5%-10%
  • Is the decline sudden or progressive ? Let’s assume it is progressive

I would split my analysis into two parts :

Internal factors :

  • Was there a change recently to the Messenger user experience ? This might have led to users getting dissatisfied with the new experience and coming back less. Here I would check with the product teams to see if there were any new releases and check with them the performance of their chosen metrics and how they evolved pre and post release
  • Was there a release recently on another product that might have created a bug on messenger’s core features ? For example, maybe users are unable to open the messenger app or unable to send messages (therefore people respond less). This might be relevant to look into since the drop was significant. We can also look into customer feedback and if we have many complaints related to messenger lately
  • Was there the same drop on other FB products (Instagram, FB, Whatsapp) ? Here we need to see if the drop is structural to the company or not (this might be caused if there are some PR issues with the brand)
  • Did we stop some marketing campaigns that were run previously ? This might lead to an increase in the previous period, and once stopped, we would notice a significant drop ==> Here I would analyze a larger period, and see if the current level of DAU is normal for periods with no marketing campaigns activated
  • Who are the users that we are currently losing ?
    • Do we notice a drop in old users ? This might point out towards a retention issue. We’ll need to look into why and how we can improve their experience to retain them more
    • Do we notice a drop in new users ? Are we acquiring less new users ? Is that an issue also observed on other FB products or just messenger ? Do we need to look into our acquisition programs ?
    • Do we notice a drop in Business related conversations ? For example, due to covid and lockdowns, a lot of businesses closed, so their customer service were no longer available on messenger

External factors :

  • Is the drop seasonal ? Here I would look into historical data to see if we notice a dip in the same period in the previous years. I would assume it is not seasonal since we were not asked to investigate this issue in the previous years
  • Is there any specific events happening around the world currently and might be impacting users’ behavior in terms of messenging ? (holidays, strikes, protests, election…)
  • Is this a market trend ? Are users using less messenging apps ? I don’t think this is likely here
  • Is there a nex competitor on the market that is rapidly growing and attracting our members ? Here I would look into the countries where Messenger’s most known competitors are based and see if we see a decline there (Asia, ME…)

Clarifying questions:

1 – Is this on mobile or desktop?  Assume it’s overall

2 – By DAU, are you looking for a specific set of user groups as there can be business users on the platform, or users in general? Assume all users are impacted

3 – Can you also correct me if DAU is defined as someone who just uses the messeneger product on a daily basis by just going into messenger, but without the need to send or reply?  Yes.  We just need a user to go into the tab or the specific app on mobile.

4 – Is this an abrupt or gradual drop? gradual, it has happened for over a period of a month and is continuing

5 – What locale is this isolated to?  No – the drop is about the same for all regions in the world

6 – Lastly, do you have a quantity for signficant?  Ah, yes it’s about 10%.

OK.  Thanks for clarifying my questions.  Let me give it some thoughts and I will come back with a list of possible causes both from an internal aspect, which is within Facebook, and also external aspects that are outside of Facebook.   Let’s see if one or more of the causes can be confirmed and I can provide a summary.  Does that work for you?  Sure.

 

INTERNAL FACTORS

OK, first off I would like to look at several internal factors which may explain the drop

1 – First of all, have we confirmed that the reports reporting the drop are correct?  Yes, the analytics tools are working fine.  No issues there.

2 – Is there any seasonality user swings historically that can be attributed to?  No.  Messenger doesn;t have any type of seasonality adjustments.

3 – Ok has there been a release about a month ago that has introduced bugs that could impact random users from using messenger?

4 – Were there any issues with the Messenger APIs?  Yes, but not all.  Our devops team has reported that the send APIs that are hosted has had some issues over the course of the last couple of weeks, but they aren’t completely down.  OK – let me note this down but this alone may not alone drive the DAU by 10%

5 – Were any other Facebook products affected?  i.e. Facebook Core/Newfeeds, etc.?  None.  They are performing in line.

6 – Were there any experiments that were run that could negatively impact Messenger?  There are experiments running on newsfeed.  OK, that may not adversely impact Messenger much as those products are pretty much separate.

EXTERNAL FACTORS

OK.  let me ask if there are external factors which may be of sigificance.

1 – Were there any social events recently like national wide protests that may have driven down the usage of the product?  As far as I know, there are small events but nothing like a massive national protests.

2 – Was there any PR written recently nefatively about facebook as a company or on our messenger product?  None as far I know.  People would occasionally ask about how Messenger works but no negative PR was reported.

3 – Was there a recent competitor launch?  Yes, Google has recently pushed a new version of Hangouts and have been promoting it in Gmail.  In fact they were prompting users to start a hangout chat and giving them beta feedback in being randomly selected for their latest PIxel devices.  OK wow,  now we are onto something.

Let me think a bit … we have an internal issue where some of the messenger APIs and that can rresult in some of the users like businesses who integrated with FB messenger cannot reach their customers on FB, and therefore those customers won’t get a message back on messenger and respond to them.  It’s sporadic but signficant enough.  Now combine that with a recent launch of Hangouts beta test with a promotion, I think I can make a sound conclusion that the combination of the messenger API availability and the Google Hangout beta test +  promo likely caused the 10% DAU drop.

Let me know what your thoughts are, or if you would like me to take action on addressing the issue.