I think the problem is that the industry hasn't created a product management layer that interfaces between non technical business folk and the technical data scientists. In software engineering, we don't expect the customers to speak directly to the engineers; that's what product managers are for (cur infamous Office Space scene).
However product managers aren't typically involved in solving data science related problems. This is primarily because most product managers don't have the math/stat/compsci background to be useful.
However I predict this will change in the next 5 years.
Agree with this. In fact, Lead Data Scientist roles often become de facto PM roles, where the LDS basically spends their time prioritizing the important research questions DS has to solve based on customer and business needs.
I've been hearing from multiple people that this is a gap that's really hard to fill right now -- PMs who can work with heavy DS and AI products. It's much easier to train experienced data scientists to be PMs than the other way round.
Agree with this. In fact, it's already changing in a couple of domains. (1) Teams that build data products often have product managers. (2) Data analytics consulting teams have project managers who interface between the clients and the analysts.
However product managers aren't typically involved in solving data science related problems. This is primarily because most product managers don't have the math/stat/compsci background to be useful.
However I predict this will change in the next 5 years.