Opinions

Product Managers and Data Scientists: The Synthesis that Drives Innovative Technologies

Posted by Kevin Reilly on April 4th, 2014 at 11:29 am

Over the past couple of decades, product management has emerged as an important function within any organization that is building technology products. Data science has arrived more recently but is also proving its value in helping to solve problems for customers. In data-centric organizations, there is an important symbiotic relationship between the two functions, but it’s not always clear where data science should fit in the product development cycle. Should data scientists be treated like developers? Analysts? Or something entirely new?

While both groups want to solve problems for customers, it is important to understand the differences in their fundamental mindsets. Data science is about exploring data, understanding its predictive possibilities and creating tools that use data to optimize defined performance metrics. Product management is about deciding if the things that are technically possible are actually worth doing from a business standpoint and if so, how those new capabilities should be delivered to customers as products or features.

Collaboration between the two groups is vital, but works best when each team has a clearly defined role and set of duties. Here are some approaches on how to successfully integrate data science into the product development process:

Data Science is the R in R&D - Data scientists need the latitude to explore problems that they think are interesting and come up with innovative approaches. Celebrate this. Not everything they do will address the problems you are currently trying to solve, but it is important to allow them the freedom to explore possibilities. Often, they will surprise you with a solution to another problem that will help shape your business in a positive way.

Data Scientists Build Great Prototypes - Projects work best when the data science team develops working prototypes using whatever tools they want. These prototypes demonstrate the requirements of a new system to software developers and others. Only once there is a working prototype that has been green lit by all the necessary parties should you move forward with building out the new capability in your production systems.

Lean Data Science is Possible - Once data science has developed a working prototype, the team should figure out the smallest increment that can be put into production to test it in the real world; a Minimally Viable Algorithm. This could be as simple as developing a workable process that provides alerts when certain conditions occur. This approach helps identify errors in assumptions and bugs in implementation before a full release that builds on the ultimate algorithm.

Data Scientists Do Great QA - I’m not referring to regression testing of code. If you are building a data product or an ETL process, a qualified expert needs to validate the data at some point. Data scientists are obviously the best candidates to conduct that oversight, so make their signoff part of the formal criteria for prototype acceptance. And after the development teams have built production versions of the prototype, don't let the data scientists wander off to get buried in the next great paper-worthy research project. Get them involved in validating that the production features are indeed working as intended.

While a lot has been written about product management, software development and how product development processes should be designed, processes that incorporate the data science function are less well understood. Data science is often essential to the development of products that can compete in a data-intensive market. Developers and product managers are best served by finding ways to integrate data science into their projects from start to finish; from design to QA and through to market feedback. Data science is now an integral part of almost every technology ecosystem, and product managers everywhere should embrace their full involvement.

Leave a comment