What metrics would you measure for a new feature on WhatsApp?

  Meta
Add Your Answer
Answers (5)

Goal/Vission of company

  • Understanding of Whatsapp -> Whatsapp platform for people where they can connect with people remotely, they can chat with them, they can do a video call, they can send pdf, location, contacts, etc. Basically, clarify your understanding of WhatsApp.
  • Whatsapp Goal -> connecting the people remotely in a hassle-free manner
  • Vission 3-4 years down the line -> More engagement and retention of the users.

For this vision what is the user’s intended action

  • Define the users and their intended actions
  • People open WhatsApp, they add the people on WhatsApp, they message them, they share the location, share contacts, share images, They also form groups, share messages in groups, They put the status.

Metrics for the actions

  • Metrics in a layered manner and have some primary metrics and secondary metrics
  • Do mention some infra metrics like latency, network error, time taken by message to get delivered, crashes, app not responding, etc.
  • Finally do mention some business metrics like the cost of acquisition etc.

Evaluate the metrics

  • People become addicted
  • Number of spams messages etc

Disappearing messages

  • Limits of # of people who can join a group call

Choosing Disappearing messages as a feature Why whatsapp introduced it

  • Updated privacy policy
  • Signal
  • Trust

Activation

  • # of users who activated it
  • # of chats/user using disappearing messages
  • Cohort analysis
    • User segment who dropped off compared to user segment who used this feature
    • churn – before and after launch

Retention

  • Avg # of days users had the feature on
  • Segment this on the time limit

Success Metrics:

  • Acquisition/Activation: # of users who activated this new feature. # of chats/groups using this feature.
  • Engagement. Avg. timeframe for disappearing messages. …
  • Retention. # of days user had this feature turned on. # of users who activated the feature but stopped using it (churn rate)

Metrics for new feature on WhatsApp

                      • Problem framing

FB mission: connect world, open. Help users have a voice (FB, IG, WA, Messenger, VR, Hardware, FB@Work)

WA JTBD: #1 most popular messaging app Rival to iMessage, Telegram, Signal and 1000s new EEE apps

  • Easy, reliable, cheap communication (messages and ‘content’)
  • WhatsApp for Biz
  • Features: appearance, privacy settings, KYC for payments, Data/ security settings, disappearing messages
  • Content
    • Media – audio, images, video, GIFs
    • Emojis, GIFs, reactions
    • Payments
    • Location
    • Documents

Biz goals

  • 2-3Bn
  • Growth/ User Acq
  • Engagement
  • Churn
  • Quality of UX (perf, crashes, errors, UI glitches)

Opportunities, pain points:

  • Competitors: Unlimited group size, Unlimited storage, Enhanced privacy features,
  • Perception battle (FB ownership)
  • New features
    • Calling from web
    • Independent from mobile
  • Search and info retrieval is difficult and time-taking
  • Content folders are a mess of valuable personal memories and transient images and forwards that have little long-term value
  • Optimization of media leads to loss of quality
                      • Solve

New ideas

  1. Search: retrieving past messages
  2. Data mgmt.: clean-up forwards, separate them out from family phots
  3. Other: Dedicated ‘Favorite’ folder for every chat thread, High res photo sharing, Continuous (ongoing) folder sharing, Send messages

Feature details Search improvements: better ML models that work with ‘poorer’ input, voice input. Different search modalities (time period, ‘address info’, <contact name>) e.g. “photos sent to Mom in May 2022”

Data mgmt.: Intelligent folder mgmt – split “WhatsApp Images” –> People / Things / Forwards / Memes (build a local client-side model to ‘learn’ from user actions) ‘Save to …’ –> Folder name

Metrics Search Improvement

  • CTR for Search results, % of ‘meaningful interaction’ (star/ forward/ copy info.) after search query
  • % search queries that are advanced (“photos sent to Mom in May 2022”)
  • CTR of % Adv Search Query

Data mgmt.

  • Pictures moved/ bulk-moved across folders: %auto-sorted photos moved
  • # of folders created on average
  • %users using ‘Save to’ functionality
                      • Summary
  • Search and Content Mgmt. two AOI
  • Metrics for both proposed above

First of all, I will define WhatsApp and understand the goal of this new feature. Assuming the new feature is ‘disappearing message’ launched pretty recently. I will then look into the customer journey map (use cases) and lastly identify key metrics to measure the success of this feature.

WhatsApp – is a messaging platform which can be used to send messages, photos, videos with individuals/groups in a hassle-free way and also used for voice and video calling along with other features such as status updates (similar to Ig Stories feature), share live location, share contacts, etc.

Disappearing Message feature was launched keeping users’ privacy and security in mind. Users can enable this feature which will auto-remove messages after the defined timeframe. The goal of this feature is to 1) improve privacy and security, and 2) increase customer engagement.

Ask Clarifying Questions:

  • Are we looking at mobile/web interface? – Mobile users
  • Is there a specific geography we want to focus on? US/Specific Country? – Worldwide

User Actions:

  • Open WhatsApp and enable the new feature and set the desired timeframe (Eg. 24 hours) for disappearing messages.
  • Send 1:1 or group messages/share content with people
  • All messages disappear after 24 hours

Success Metrics:

  • Acquisition/Activation:
    • # of users who activated this new feature
    • # of chats/groups using this feature
  • Engagement
    • Avg. timeframe for disappearing messages
    • # of chats/groups per user where this feature was used
  • Retention
    • # of days user had this feature turned on
    • # of users who activated the feature but stopped using it (churn rate)

To summarize, the goal of this feature is to provide a secure platform for users to send messages and share content with others, build user trust and improve customer engagement.