First I would like to ask some clarification question-

What does It mean by video engagement ? are we referring to total hours of video watched or  we referring to total number of clicks on video.

Let’s take the first one.

Restating the question-we have noticed a drop of 15% in total hours of video watched over the past two weeks. As a product manager what will you do?

Clarification question-

1.is it happening on a specific platform desktop, mobile (android ,apple)?

2.is it happening in a specific country?

3.is it happening for short video(30 seconds video) or long video?

4.was there any external region such as bad PR?

5.was there any internal reason such as bug or server issue?

6.were we doing any experiment such as A/B testing?

7.have we changed metric calculation formula?

8.has any new competitor came in the market?

9.what about existing competitors? have they launched any new feature? Is their market size is growing?

Based on these clarification questions we can know the potential cause and thus give solutions for it.

Structure:

1. Clarifying questions
2. Diagnose the problem
3. Propose solutions

Clarifying Questions:

1. How do you define engagement on the platform? Is it number of mins viewed per customer has gone down or the total number of mins stream on YouTube? Total number of mins streamed on YouTube has gone down.
2. Looks like this is a sudden drop of 15% Over the past two weeks. Has this happen in any other month this year? Also, did we see a similar drop same time last year? No
3. Was there any data logging issues with how we report the metric? No
4. Have we changed the way we calculate the metric? No
5. Is this drop related to any particular Geography? No
6. Did we launch any new feature recently that could affect the engagement? Perhaps we didn’t include engagement in our Guardrails metrics for the new feature launch? No

Diagnose the problem:

External factors: Regulation, Competition, Market, Economy.

1. PR related issues: Did we have an negative PR recently? No
   1. Any recent data leaks that led to loss of customer trust? No
2. I believe we have many competitors both direct and indirect. Do we know if a similar drop happened at the competitors’ platform? Not that we know of
   1. Direct: other streaming platforms with long form content: Netflix, Prime Video etc.
   2. Indirect: Instagram video, TikTok, etc.
3. Have we seen it on a particular device? No
   1. Browsers: Chrome, Safari?
   2. OTT: Apple TV, Roku, Fire TV
   3. Smart TVS: Samsung, LG
   4. Mobile Devices: iOS, Android?
4. Have we pushed a new App version across devices that led to the negative impact on the engagement? No
5. Is there a new competitor in the market that competes directly with Youtube and is taking the market share? No
6. Was there a big event in the last two weeks such as GOT season Finale that could have affected the engagement? No

Internal Factors: Customer, Product, Company.

Company:

1. Company goal has changed? No
2. Did we lose engineering resources or head count? Internal Strike? No
3. What is company optimizing for? Engagement? Revenue? No, the focus is still the engagement.

Customer:

1. Viewers:
   1. Did we see any drop in the daily or weekly active users?
   2. Did we see any drop in the net new users? Across any geography?
   3. Did we see a churn in specific type of the user segment such as skill builder, entertainment seeker.
   4. Have the number of users sessions per day/week have gone down?
2. Content Uploaders.
   1. Did we see any drop in the daily or weekly active users?
   2. Did we see any drop in the net new users? Across any geography?
   3. Did we see churn in any particular segment? Studio content uploaders vs freelancers?
3. Advertisers
   1. Has total number of impressions changed? gone up or down?
   2. Is there a churn in advertisers? i.e. are they leaving YouTube and going to other platforms.

Product features:

1. Uploading Content.
   1. Any information around drop in content for any specific length?
   2. Any information around drop in content for an particular language?
   3. Any information on drop in any particular type of content?
      1. Podcast,
2. Recommending Content
   1. Any change in how we recommend content to the users?
   2. Any change in the YouTube homepage that shows content to the user?
3. Streaming experience:
   1. Playback experience is still good? No buffering etc.
   2. Has the in-stream ads gone up?
4. Engagement with content.
   1. Has there been any drop in the likes, shares, and comments of the video?

Tech Infrastructure:

1. Availability
2. Scalability
3. Reliability
4. Speed to fulfil the request.
5. Load handling.
6. 3P integrations are working fine.

Clarifying Questions :

1. How do we define engagement? Is it based on engagement on video creations/sharing, consumption or both? Is there a minimum amount of time required while watching a video /or any other prerequisite before it’s considered an engaged user?
2. Is this drop a sudden drop or gradual? Is this a one-time occurrence or is this a recurring issue? (One-time drops can be a symbol of a technical issue, while if the drop is progressive it’s likely a larger issue that requires deeper investigation)
3. Is this drop in a specific region globally distributed? (If it’s localized to a specific region we could look at changes to the region in the past two weeks, or look at technical problems associated with the region)
4. And in a specific platform –desktop, iOS/android?
5. Are we seeing a drop in other features or products within our ecosystem? (If this drop is happening across other features and products, it’s likely a larger problem we should address with multiple teams)
6. Was there any new feature that was launched in that platform in the last two weeks?
7. Is there any new competitor’s in the market?

Assumptions:

Less comment, post/sharing.

New competitors taking market share.

Identifying Pain Points:

Is the distribution of the decrease uniform across all videos. The users did not enjoy the long videos/

Is the model or suggesting videos outdated?

Has the comment posting changed in any way?

The user’s had better content/compelling content on a competitor’s platform.

Solutions:

Give feedback to content creators to not make videos more than 30 mins.

Have a check to not upload videos more than 30 minutes

Revamp the recommendation engine

Do a competitive analysis-give a free 3 months subscription for new users and existing users

Partner with a platform to show things on youtube