How To Measure Product Adoption - Issue 27

Product adoption, data democratization, and DS trends in 2021

Hello analysts and welcome to a once-a-month-free-edition of the Data Analysis Journal newsletter, where I write about data analysis, data science, and business intelligence. 

If you’re not a paid subscriber, last week you missed a User Activity Analysis in SQL (one of the most common SQL problems often asked on technical interviews) and my walkthrough on analyzing user activity by applying SQL RANK() function.  

✨ In today’s newsletter:

• How to deploy ML models to production and why 87% of them never make it.

• How to measure the success of a new feature or a product.

• What data democratization is, why it is important, and how to develop data culture at your company. 

• SQLserver 101 or how to get started with SSMS basics. 

• 2021 digital marketing trends report.

  *For paying subscribers, apologies if you received this newsletter twice. Big mistake happened while figuring Substack magic.

  

  🏆 Nailed It

  Be prepared for your next interview

  How to measure the success of a new feature release or run a product adoption

  One of the most common challenges in product analytics is measuring the success of a new feature release or analyze product adoption - how users discover it, how they use it, and how often. Simple questions can become quite complex when you start breaking these down on a timeline, user segments, or engagement layers. 

  For subscription-based products or services, measuring product adoption is essential. A high adoption rate leads to increased: 

  • revenue 

  • LTV

  • trial to subscribed conversion

  • free to paid plan conversion 

  It’s a lot of pressure on ensuring you tackle the measurements correctly. Here are the top 2 metrics to start with: 

  1. Adoption rate:

  % users who used a new feature / all existing users 

  This metric is useless without applying a specific timeline. To get the Adoption rate for December, users who used the feature for the first time any day in December divide by the total number of users you have, on the last day of the month.

  2. Time-to-Action:

  avg time for a new customer to use an existing feature

  or

  avg time for an existing customer to use a new feature for the first time.

  ❗Things to remember:

  • The more users are engaged with a product, the faster they will discover new features, and the more often that they will use them. This is a big caveat if you are running an A/B test for a not randomly distributed audience. The measurement of feature discovery and usage will be skewed and potentially overreported.  

  • If your feature of the product is complex or requires customer learning, you might have to introduce the Time-to-Adopt metric in addition to the Time-to-Action to measure how much time it takes for users (either new or existing) to start using a new feature.

  • Another secondary metric to look at is Duration - how long users keep using the feature after discovering it.

  If you are launching a new product and working on setting top-level metrics, here is a good guide from ProductLed in order to best understand your Northstar metrics and KPIs measurement.

  🎓 Level Up

  Refine your skills, develop, and grow

  Today, I’d like to share today a new publication from Data-led Academy about developing data culture at your company and how to promote data democratization - “enable teams to work with data, feel comfortable talking about data, and as a result, make data-led decisions.”

  Here are some interesting takeaways:

  The article covers different ways teams are using data for:

  • Marketing works with data to create engaging, better-converting content 

  • Growth works with data to run experiments and deliver personalized experiences 

  • Product and Engineering work with data to build features that are actually used and kill the ones that are not

  • Support works with data to deliver faster resolution (by seeing what a user has done or not done inside a product)

  • Customer Success works with data to deliver a better customer experience (by asking them the right questions based on usage patterns)

  • Sales works with data to identify prospects that are likely to convert (by looking at the actions they have performed during the free trial)

  The most common data challenges include not having the access to the data, lack of trust in data, no appropriate skills to find the answer to questions, not robust or helpful analytic tools, or simply that other data experts are too busy to help. 

  Read What is Data Democratization? to learn how to tackle these challenges and be an important part of creating a cultural shift at your company towards data democratization. 

  🔥 What’s new this week

  2021 Data Science Trends report 📈 covers DS trends we should expect to see in 2021:

  1. Operations - DataOps and MLOps which are responsible for bringing AI models and theoretical results into the production.

  2. Data democratization - technologies and services that make data easily accessible to everyone in the company for decision making.

  3. Hyperautomation  - combining AI, analytics, and IoT that augments Robotic Process Automation (RPA) to automate business tasks and improve efficiency.

  4. Low-code no-code platforms - automated ML services developed for non-technical experts.

  5. Cloud-native ML - flexible architecture that “scales according to its usage.”

  6. Explainable and responsible AI - regulations on how AI is trained and developed, making it more ethical and trusted. 

  2021 Digital Marketing Trends report 📈 does a good recap of 2020 with setting a projection of some trends for digital marketing:

  • Video streaming increased by 12% in the first week of quarantine in the U.S.

  • In March 2020, when the lockdowns and social distancing protocols first began, Facebook’s analytics department reported a 50% surge in messaging. Similarly, WhatsApp saw a 40% lift in usage” And you know the story with TikTok.

  • In 2021, people will spend an average of 100 minutes per day watching online videos.

  • Marketers will likely continue to pursue deeper personalization and more localized targeting — trading reach for relevance.

  PostgreSQL is the DBMS of the Year 2020 🎉

  The second place is the SQLserver (Azure SQL DB built on top of SQLserver). The third place is MongoDB. 

  Basics of SQL Server Management Studio (SSMS) – Part 1 ⚒️:

  Basics of SQL Server, how to install it and get started. Must-know terminology, prerequisites, and basic actions.

  📚 Weekend Longread

  How to put machine learning models into production

  The goal of building a machine learning model is to solve a problem, and a machine learning model can only do so when it is in production and actively in use by consumers. As such, model deployment is as important as model building.

  BUT. 87% of data science projects never make it to production.

  There can be a “disconnect between IT and data science. IT tends to stay focused on making things available and stable. They want uptime at all costs. Data scientists, on the other hand, are focused on iteration and experimentation. They want to break things.”

  Bridging the gap between those two worlds is key to ensuring you have a good model and can actually put it into production.

  🍸 Drink and Mingle

  Upcoming free events, meetups, talk, and webinars.

  • Jan 13, Galvanize: Intro to Python: Part 2

  • Jan 14, WWC London: ML Workshop: Hunting for Pulsars

  • Jan 14, Redis: Introduction to Redis for Data

  • Jan 18, Galvanize: Intro to SQL and RDBMS

  • Jan 21, Recurly: How Finance and Operations Leaders Can Use Data to Unlock Growth

  • Jan 21, Galvanize: How to Approach a Python Coding Challenge

  • Jan 26, How To Build an Interactive Power BI Report

  

  🙏 Do Some Good.

  It pays back.

  I am one of the members and volunteers of Women Who Code (WWCode) - the largest and most active community for technical women in the world. WWCode award-winning programs support over 230,000 members around the globe to level up in their technical careers, every step of the way. They have produced 10,000 training, workshops, hackathons, conferences, and more. If you are looking for an organization to donate to, please support WWCode work. Donate here.

  Thanks for reading, everyone ❤️. Until next Wednesday!