first, some clarification questions:
1.  let us validate the goal of the youtube search – the goal of the search feature is to help users discover relevant videos and content by providing a user-friendly and efficient search functionality. correct? {answer : thats right}

2. at what stage are we of this feature? are we launching it for the first time? or is it already implemented and used. {answer : it’s already in use for couple of years}

3. should the metrics refer to specific type of users? (children, elders, specific geographical region)  {answer: it reffers to everyone}

4. what do we want to learn about the feature? the search efficiency (speed/accuracy of results)? is it’s design friendly and convenient?   {answer : we need metrics that will indicate if the search is efficient}

Given the answers above, i now turn to simply describe the important parts of the users search journey on youtube:
fill the search bar at the top with description of the vontent im searching (using keyboard / record to text) -> click the search button -> see a list of results sorted by relevancy to my search -> either click on a video form the list (refers also to clicking on the ‘next pag’ button) OR changing the text in the search bar and accurate the text im searching videos by.

Having that in mind, these is a list of signals i need to form goot metrics:
general
1) # of searches a day

accuracy of search
2) result depth – the result number from the top, when 1 is the first result, 2 is the second result etc.

3) following searchs – number of searches without clicking videos result in between
4) conversion rate – the user actually watched the video (watched more than 25% of the video or at least 30 seconds, the shorter one)

Speed
4) The time passed from the ‘search’ click until results are pulled from the backend

(if the search is slow, we can make a seperate count for the time passed from the ‘search’ click until results are pulled by backend and the time are displayed on the screen)

Now lets define metrics using these signals:
* % of accurate sorting – search with click on result 1-3 / total number of searches
* % of accuracy – search with click on result AND conversion rate / total number of searches
* Avg time for a search per day – total time of all searches / # of searches
* Avg number of searches needed to find a result

Clarifying questions:

Youtube search – Product that allows for users to search for content by entering keywords in the search bar.

Platform – WWW and mobile

The goal of the product: 

To allow users to search for new content by allowing users to convey their intent to Youtube and find relevant videos.

Key users of search:

1. Text Searchers
2. Voice searchers

Key Actions of 

1. Text Searchers – Enter the letters, words in the search input
2. Voice searchers – Input voice commands
3. Clicks and browses of the search results
4. View video after clicking on the result

Key metrics that capture the above

1. # of text search inputs
2. # of voice search inputs
3. Scroll depth (to understand how much a user scrolls after they enter their query)
4. Click-through rate on search results
5. Bounce rate to make sure the result is not high

Evaluation:

1. I would like to see how the directional change in these metrics have some impact on user metrics like DAU and WAU. Example If we see a lift in  #of search entries, CTR we should see a similar directional lift in DAU.

Structure  

I will review YouTube’s mission, make sure that I understand the product and the users it serves, and ask for some clarification questions. Then, I’m going to select a goal and list the potential metrics.

The missionAllow users to watch and share videos – give users a voice and show them the world
The ProductThe Search is one of the site’s main features that allows users to explore content; Some users immediately use it once landing at the homepage, while others look for recommendations. Sometimes users use the search multiple times during a session. After searching, there are also options to sort filter the results.

Types of search:

1 – I know exactly what I’m looking for

2- I think I know how to spell it

3- Not sure what am I looking for

Clarification questions

Should I focus on a specific users segment? Device category? Should I focus on the search or also on the filters and the results? Should I focus on the search from the homepage or from other pages. – I’m going to focus on the search from the homepage, including the interaction with the result page.

Goals

I think that the most interesting goal related to the search is engagement; Will users be able to find what they are looking for, or maybe something even better. And how much effort it will require.

Metrics North Star Metric

How many sessions that used the Search at the homepage eventually led to watching a video. How many searches and how many clicks it required?
Definitions:
Watching a video = completing at least 70% of it
Click = open a video from the search results.

Secondary Metrics

# MAU that search

# of searches

average searches per user

Counter Metrics

# of searches with a “few” or no results
#/% of searches without a click that led a user to abandon

– Performance of the search results (page load, time to return the first result, time to provide a suggestion,..)

Clarify the question

Why do we want to measure the metrics? For example the answer could be we want to find devices where search is not performing as expected (or) it could be that we want to measure how search is performing.

Youtube is available in multiple devices.

Goal of the search product

The goal of the product should be to enable the user to find the most relevant video with minimum interaction.

User Journey

User search for keywords using text or voice

User uses the suggestions or auto-complete

User browses the videos in the search results.

User views the videos typically displayed on top

Metrics

The most important part of the search is how users will be able to find the most relevant search results with minimum effort. Most of the users typically click on the first 3-4 links and then stop browsing.

The effectiveness of the search needs to be measured by qualitative studies and quantitative means

Measuring for metrics can be done thru user studies (by measuring how the users are querying, how relevant are the search results). Unfortunately, human interaction is required to measure search relevance and positioning of the search results.

Some of the quantitative metrics we can look at:

1.) Time spent for user to search (voice, text)

2.) Rate of user of suggestions / no. of searches (voice, text)

3.) Click thru rate

4.) Conversion rate (The user actually watched the video)

5.) Mean rank of the search clicked (this will give input on the positioning of the video in search results)

6.) Bounce rate (user did not click any videos)

7.) Load time of the search result page

We can also slice the above metrics on time period, geographical location, device, browser etc.

The combination of qualitative studies and quantitative data can be effectively used to measure the performance of YouTube search.