Showing posts with label product manager interview questions. Show all posts
Showing posts with label product manager interview questions. Show all posts

Important Metrics for Product Managers

There are 3 broad categories:

User engagement 


User Engagement metrics: Use cohort analysis over a period of time. This will help understand how the behavior has changed overtime. 
  1. Number of sessions/user. 
    1. How often the users log in or open the app. 
  2. Session duration for a cohort(group of users performing a specific function) over time:
    1. How much time they spent interacting with your product. 
    2. Compare average session duration between churned and retained users. 
  3. Number of key user actions/session
    1. Select important user actions(like click on something) then trace them over time and for different groups of people. 

Business metrics

  1. Customer Lifetime Value - LTV
    1. Amount of revenue generated by different customers. 
    2. Don't calculate Customer Lifetime Revenue. 
      1. Deduct operating and development costs. 
  2. Customer Acquisition Cost - CAC
    1. Acquisition marketing costs/number of paying customers acquired over period of time. 
    2. Acquisition costs = all operating and dev costs. 
    3. For healthy SaaS startups LTV/CAC >=3
  3. Average Revenue/Account - ARPA
    1. Amount of revenue per account per month. 
    2. If this metric is stable then team is on right track to provide value to customers that keeps them on the platform.
    3. If you have better support and branding then you can consider increasing ARPA without changing pricing or product. 
  4. Monthly Recurring Revenue - MRR
    1. Company's or product revenue/month. 
  5. Logo churn & Revenue churn rates:
    1. Percentage of paying customers/account that are lost/month. 
    2. Calculate as: cohort and aggregated. 
    3. Revenue churn: Revenue the company loses/month due to churns or downgrades. This is important. 
  6. Retention rate: 

Customer Service:

  1. Number of incoming support tickets: Smaller the number better the state of the platform or software. 
  2. Net Promoter Score
Source: https://productcoalition.com/critical-metrics-every-product-manager-must-track-c5f1e46e3423

Some math basics for metric calculation:

  1. Mean = usual average of all the numbers
    1. Ex: Mean of 1,2,3,4,5 = (1+2+3+4+5)/5 = 3 
  2. Median is the middle value of all the sorted numbers from the input sources:
    1. Ex: 1,2,3,4,5: Median =3 
    2. Ex: 1,2,3,4: Median = (2+3)/2 = 2.5
  3. Mode is number that is often repeated:
    1. Ex: 1,3,3,3,4: Mode = 3
    2. Ex: 1,3,4,5: Mode = none since none of the numbers are repeated. 
  4. Range: Difference of smallest and largest number
    1. 1,2,3,4,5 = 5-1 = 4
You might also like:

Media Ad Concepts

Personalization concepts:

There are many categories of web personalization including

  1. Behavioral
  2. Contextual
  3. Technical
  4. Historic data
  5. Collaboratively filtered
There are several camps in defining and executing web personalization. A few broad methods for web personalization may include:
  1. Implicit
  2. Explicit
  3. Hybrid

---------------------------------------------------------------------------------------------------------------------------

Media Ads Concepts

  1. Ad inventory: The number of ad spaces available with the publisher. If a publisher has 100 spots, where s/he can display the ads then 100 is ad inventory.
    1. It is valued in terms of the site traffic or ad views that can be delivered to the advertiser.
    2. It is calculated in number of page impressions. If site A has 100 impressions/day and site B has 1000 impressions/day then site B can ask for more money from advertiser because site B can reach to many websites. 
  2. An ad value can be calculated as page impressions X average number of ads on your website pages Although above formula is simplest of all. 
    1. Moreover, it can be calculated by the location of the ad. Ads displayed at the top of the webpage demands for more money since it is the prime stop as compared to ads displayed at the bottom. 
  3. Fill Rate: It shows how much of your ad space has been rented out and how much of it is empty. Higher the fill rate better are the chances since it means your website is in high demand.
  4. Revenue: A website’s revenue can be calculated by number of leads or actions are performed after clicking on the ad or after cpm(cost per 1000 impressions)

---------------------------------------------------------------------------------------------------------------------------


Metrics Concepts:

Here is the summary of articles on Product Metrics. Article: Only Metric that counts
Summary: 
  1. Are people using the product? Once you find that out, find out:
    1. Why do they use your product? 
    2. How frequently? 
    3. What key actions they perform while using it? 
  2. Answers to above questions will tell you who are core users, who:
    1. Come directly
    2. Return weekly/monthly
    3. Share it with others
  3. Reference:
    1. https://news.greylock.com/the-only-metric-that-matters-now-with-fancy-slides-232474cf414c

---------------------------------------------------------------------------------------------------------------------------

Ad Ecosystem:

This section will explain you the basics architecture of ad ecosystem. Definitions of different components and how those components work together to render ads:


Publisher:
  1. Runs online business or websites that has user traffic.

Ad Network:
  1. Aggregator of publisher ad inventory that is packaged by contextual channels, audience types or buying options(CPC, CPA, CPM). This inventory is sold. 
  2. It is an entity/company/organization that connects advertisers to websites.
2.     It enables to deliver ads to consumers, perform targeting, tracking, and reporting of impressions.
3.     Inventory of online ads include:
a.     Blogs
b.     Rss feed
c.      Web pages
d.     Mobile pages
e.     Instant messages
f.      Mobile apps
g.     Adware Emails
h.     Other media
4.     A publisher(new york times) can integrate with an ad network. Ad network can then display certain ads
5.     Large publisher sells remnant(not sold = 10-60%) inventory through ad networks. Smaller ones sell everything through the ad networks.
6.     Large ad networks include:
a.     Search engines
b.     Media companies
c.      Tech vendors


Seller Side Platform:
  1. Platform for publishers to manage ad inventory and enables routing to different ad exchanges


Ad Exchange:
  1. Enables buying and selling of Media ads from multiple ad networks.A marketplace for buying and selling of ads. Ad inventory is auctioned to advertisers/agencies.
  2. Handles Real time Bidding for publisher that includes:
  3. -       Purchases inventory at individual impression level
  4. -       Bidding in real time(milliseconds)
  5. Ensures that publishers can maximize value of their inventory.
  6. Ensure that advertisers purchase individual impressions at fair price.            

DSP:
  1. Platform that enables media and audience buying for advertisers in real time on ad exchanges    
  2. Evaluates and bids on each individual impression through RTB based on available user data.
  3. Takes care of RTB for advertisers.
  4. Centralized campaign management, reporting, optimization to meet advertiser’s objective.

RTB process:
  1. Publisher puts the available inventory on the ad exchange
  2. Ad exchange holds auction for each impression
  3. DSP receives the bid from advertisers.
  4. DSP evaluates the impression and calculate bid amount based on its value to advertiser. 
  5. Ads from winning bid are shown on the publisher’s site.

Advertiser:
  1. Spends on marketing to grow their business.

---------------------------------------------------------------------------------------------------------------------------


More resources:

  1. 5 things ad tech product managers should consider when planning their company's future
    1. NOTES:
      1. While contacting SSPs lot of information like user browsers, location, etc can be collected from requests & processed auctions.
      2. User information can be collected via the DMP. They will have services that will provide you user information via cookie syncing services.
      3. Consider inventory types like TVs or VRs where ad can be served in future. Product requirements should take care of such things. 
      4. Consider use of machine learning algorithms to generate metrics that will help describe if correct data is collected and if right metrics are generate. 
      5. TODO: Look at new video ads protocol
      6. Make the product flexible to allow for change in trends
      7. Cost of scale: Don't let technology costs kill the when scaling comes into picture.  
  2. Confessions of an adtech product manager

Product manager design question #2

How would you implement Google Chrome for Kids navigation.

As usual we will tackle this design question with keeping Goal in mind. We will then list out the personas, their use-cases, features, metrics to track, and prioritization.

Goal: Safe browsing for kids.

Personas:
  1. Toddler
  2. Young kids
  3. Teenagers

Use-cases:
  1. Education
  2. Browsing
  3. Entertainment

Solutions/Features
  1. Profiling: While setting up the browser the parent/authority will set the profile in a manner so that safe browsing is always enabled. Payments are automatically disabled in this profile.
    1. Pros: Serves the purpose
    2. Cons: Extra setup and some technical expertise.
  2. API call:
    1. When call is initiated from the safe profile then safe results are surfaced.
    2. Payment fails if the call is initiated from the safe profile. This ensures that child doesn't order digital products by mistake.
    3. Pros: Serves the purpose.
    4. Cons: Requires expertise as to which content makes sense v/s which does not.
  3. Notification: If weird/questionable content is requested then notification is sent to the authorities for review.
    1. Pros: Proactive notifications ensure parents are on top of their kids searches.
    2. Cons: Chances of false positive. If kids is required to do some weird searches for their biology class then it will result into false positives.
  4. Override for parent functionality wherein they can use the same browser for their use.
    1. Pros: Convenience. No need of second browser on same machine.
    2. Cons: Chances are kid could learn how to override defeating the whole idea.
  5. Reporting a summary of their kids searches.
  6. Mature APIs based on search results and customer satisfaction.

Metric:
  1. Customer satisfaction: Need to ensure that parents are satisfied with the product. And children are able to use it with ease.
  2. Implementation complexity: Based on customer satisfaction, we will prioritize features based on implementation complexity. High complexity features can start earlier. Features on which a lot of other features are dependent can start earlier.

Prioritization:

  1. Profiling as it a basic feature & API call as its part of the basic feature as must haves in MVP.
  2. Override is also MVP as otherwise the browser will be unusable for the parents.
  3. Notification, Reporting, Mature APIs can be part second release.

Product manager design question 1

How would you implement Chrome for Kids navigation.


Background:
Safe browsing for kids.

Personas:
  1. Toddler
  2. Young kids
  3. Teenagers
Use-cases:
  1. Education
  2. Browsing
  3. Entertainment

Solution: Metric: Implementation complexity:  
  1. Profiling: While setting up the browser the parent/authority will set the profile in a manner so that safe browsing is always enabled. Payments are automatically disabled in this profile.
    • Pros: Serves the purpose
    • Cons: Extra setup and some technical expertise.
  2. API call:
    • When call is initiated from this profile then safe results are surfaced.
    • Payment fails if the call is initiated from this profile.
    • Cons: Requires expertise as to which content makes sense v/s which does not.
    • Pros: Serves the purpose.
  3. Notification: If weird content is requested then notification is sent to the authorities for review.
    • Pros: Proactiveness
    • Cons: Chances of false positive. If kids is required to do some weird searches for their biology class. It will trigger a meeting with parents/school authorities.
  4. Override for parent functionality wherein they can use the same browser for their use.
    • Pros: Convenience. No need of second browser on same machine.
    • Cons: Chances are kid could learn how to override defeating the whole idea.
  5. Reporting a summary of their kids searches.
  6. Mature APIs based on search results and customer satisfaction.

Prioritization:
  1. Profiling as it a basic feature & API call as its part of the basic feature as must haves in MVP.
  2. Override is also MVP as otherwise the browser will be unusable for the parents.
  3. Notification, Reporting, Mature APIs can be part second release.

Product management: Marketing Frameworks: Cheat Sheet


During Product management interviews a lot of interviewers ask 'How will you market a product?' Reason is a lot of positions require interaction with the marketing teams, hence the question. For these type of questions, its best to answer with some frameworks. Here are 3 such frameworks to help construct your answers.

  • AIDA
  • REAN
  • Marketing mix – 4Ps

AIDA:

This framework is useful for convincing the customer to buy your product. Here is the description of each of its phases

  1. Attention: Get customer’s attention.
  2. Interest: Get them interested in your product by telling them how it is going to solve the problem.
  3. Desire: Make customers want your product by telling them how it will help them.
  4. Action: Enable them to purchase the product.


REAN:

This framework is useful to maintain the relationship with the customer and keep them engaged so they remain your loyal customer.


  1. Reach: Reach out to existing or new customers.
  2. Engage: Engage with them by understanding their needs and relate it to the product features.
  3. Activate: Motivate customer to purchase your product.
  4. Nurture: Keep engaging with them to know their pain points and solve it at regular intervals.

Marketing mix – 4Ps


  1. Product: 
    1. What are the product’s features and how does it caters to the customer’s needs. 
  2. Price:
    1. Determines the customer personas who are buying the product or willing to buy the product. 
    2. How many customers purchase the product.
    3. Different prices for different channels like online, offline, retail stores, door to door marketing, etc
  3. Promotion:
    1. All forms of Ads, PR, word of mouth, sales,
  4. Place:
    1. Online or offline or at home. For ex:
      1. Online: Amazon or ebay or retail site
      2. Offline: Physical store, kiosk at mall
      3. At-home: Gym instructor coming to home for personal training.

Hope you liked this cheat sheet as it will come in handy for the interview process. If you want cheat sheets on more topics, then please comment down below. I will be happy to create them for you. 

NoSQL

NoSQL Data models: key-value  Aggregate model.  key or id is used to get the data.  Lookup is based on the key.   document Aggre...