Observations on the drivers of success and failure
Imagine you’re about to have heart surgery. Your surgeon drives to your house, bathes you, drives you to the hospital, preps you for surgery, administers anesthesia, performs the surgery, moves you to the recovery room, and eventually drives you home. Crazy right? This is exactly the life of a data scientist.
This situation compounds the cognitive dissonance between business leadership and their data science teams. Leaders read how Google, Uber and other Digital Native companies generate value from ML and set those same expectations for their organizations. The results never match the expectations and leadership does not see meaningful business impact from ML. They also quickly understand that things take way longer to deploy than is expected; that there appears to be an endless appetite for more ML talent, and that the answer just seems to be hire more data scientists.
On the other side of the conversation, the Data Science teams are having their own cognitive dissonance. They joined their organizations with the promise of doing impactful ML work; instead they spend the majority of their time acting as data janitors; tediously stitching together the ML pipeline manually. Much of their best model work never sees the light of day.
These people entered the field because they are interested in research and become energized by solving problems and having impact. They become demotivated by repetitive and boring tasks and don’t see the value in using their time to make existing systems more efficient. They’re not wired to be engineers. The problem is 80% of building and deploying an ML model is the exact plumbing of the end-to-end ML Pipeline they’re not built or motivated to handle.
That’s where the double whammy comes in!
Management becomes disillusioned with the promise of ML versus what they are seeing. Instead they focus on time and expense without any tangible impact for it and begin to doubt that ML will ever drive value for the enterprise. The Data Science teams become disillusioned by the work they are doing and by the way management is viewing their work. These are very valuable human resources who are constantly sought after. Disillusionment quickly turns into a retention problem. Loss of key members of the team further reduces the effectiveness of the ML programs, confirming management’s suspicion that the efforts will never pay off, further disillusioning the data science team.
Soon it becomes almost irrecoverable. Management loses all credibility in ML potentially being a game changer for their business, and the data science community comes to think of the organization as a toxic place to work. I know this can sound a bit dramatic and extreme; but it’s real. I’ve witnessed this exact cycle in some of the organizations I’ve worked with.
This does not need to be the reality. The key to avoiding this cycle is to free the data science team from the 80% of ML work that is plumbing. This will unleash the organizations most valuable data science resources to deliver on the promise of the ML program, driving both value for the enterprise and satisfaction for the human talent. There are technologies available today, that coupled with the addition of data engineers, can free up data scientists from the repetitive tasks that sap their value; and automate many of them. KubeFlow is an excellent example of what I’m referring to.
Just applying this type of insight on nature of ML tasks and talent to ML initiatives can quickly turn the cycle of disillusion to a cycle of success in the enterprise. With data scientists unleashed to do what they do best, solve complicated problems in highly innovative ways, ML initiatives can begin to generate meaningful impact on the enterprise, satisfying management’s hopes and expectations. The data science teams feel fulfilled and rewarded by the work they are doing; in an environment where they are driving genuine innovation. The enterprise both sees results and becomes a place that keeps and attracts the very talent it needs to see the continuing benefit of its ML programs.
It may sound overly simplistic but it’s not. ML is a complicated universe for every organization. A key to the success of any ML program is unleashing the talent and human capital the enterprise has invested in. That takes investment in the ecosystem that surrounds them to allow them to thrive and innovate.
About the author:
Laks Srinivasan has 15+ years of executive management experience in global organizations that help enterprises harness AI and ML for concrete, measurable business results. He has held a variety of new product creation and operating roles in leading data analytics companies. Through dozens of real-world customer projects across a portfolio of verticals, Laks has a wealth of practical experience in deploying AI and ML at scale from concept to commercialization. He is skilled in strategy, software and organizational execution required for value realization of AI and ML. Laks has worked extensively with clients in government and commercial sectors to transform Marketing and Sales, Customer Intelligence, Operations, and Risk/Fraud management functions.
Laks holds an MBA from Wharton in entrepreneurial management and finance, as well as a BS in electrical engineering from NIT, India. Laks is a senior fellow for leadership development at the Enterprise Systems Center, Lehigh University where he mentors senior and graduate students on data analytics projects for industry and government.