Artificial Intelligence is slowly becoming our norm with Alexa and Siri acting on our commands, Uber and Lyft driving us around and Netflix and Amazon lounging in our entertainment arena. Even with the growing adoption across various sectors, enterprises are finding it hard to build AI projects aligning with business goals. As per the MIT Sloan Management Review, only 23% of enterprises have fully adopted AI.
Working with small and large enterprises on multiple AI projects, we tried to compile a list of questions by observing enterprises that have successfully executed on AI projects.
1. Are your stakeholders aligned on the expectations from AI?
AI projects involve multiple stakeholders starting from the business and product team strategizing goals to Data Scientists, engineers and operations teams building and operationalizing the models. While big data technologies have provided a means to capture all the nuances of enterprise business processes, Artificial Intelligence and Machine Learning provides a way to extract actionable insights from it. However, given the nascency of the field and the multiple moving parts in the data and intelligence pipelines, it is important for all stakeholders to approach AI with the right expectations – start small, learn and expand.
2. Do your Data Scientists approach AI with a business outcome-first attitude?
Being a Data Scientist is as much about science and statistics as it is about solving a business problem. Data Scientists should focus on understanding the entire business process and work with business stakeholders before attempting to solve it. Even if you are able to achieve 99% AUC (Area Under Curve) for your classification problems, perfect MAE (Mean Absolute Error) and MSE (Mean Squared Error) for regression problem, if your model does not fit well with the business process and can not be utilized by business owners, there is very little use to such a data science project.
3. Do you have access to the right data and understand its quality?
There are gazillions of data moving around but building effective AI necessitates a high-quality dataset. Data is the main culprit for majority of AI failures. Either the data is missing, or of low quality, or not in the right format, any of which makes the cost of data acquisition and processing quite high. It is advised not to approach an AI project without a good understanding of your data, domain experts and data engineers that can work with it to help the modeling process.
4. Do you have a highly collaborative environment for your interdisciplinary team?
Any business undergoes frequent changes either in the process or data flow resulting in a model that is no longer relevant. Without cognitive collaboration among data scientists and the ML Ops team, all these tasks to build and deploy models become mundane eventually.
The arrival of AutoML allows not only data scientists but anyone working with data to build models automatically without knowing the intricacies of algorithms. It is welcoming to see multi-disciplinary collaboration involving not only engineers and data scientists but also people from all walks of life contributing to the AI revolution.
5. Does your team have a data science development process?
There is a spur of growth in Data Science granting Data Scientists the freedom to choose any ML stacks or libraries. With freedom comes the responsibility to choose the right libraries for the right problem.
The build phase includes extensive steps starting from data collection to model versioning. The iterative nature along with extensive steps to build the ML model makes a failure, even more, costlier if not acknowledged sooner.
6. Do you have the operations support to take models into production?
The iterative nature of AI projects combined with the unpredictable model behavior in the real-world tightens the commitment towards AI. Employing AI without proper infrastructure and good literature on the problem leads to an illusion that AI is the reason for failure.
The evolution of Data Science is mandating the need for new interdisciplinary talent pool knowledgeable not only in building models but also in the deployment process. Even if an enterprise employs the operations team for deployment, there are few choices of tools that understand data science and it is even harder to find talent with both data science and ops skill sets. Added to that, machine learning as a discipline continues to evolve quite rapidly with newer algorithms (deep learning, reinforcement learning) and frameworks that have varied operations challenges for both cloud and on-prem.
7. Do you have a strategy for maintaining AI, after it has been deployed?
Building effective machine learning models involve frequent experimentation with different algorithms and ML libraries followed by continuous retraining of models. Even if you are able to achieve 99% AUC (Area Under Curve) for classification problem, perfect MAE (Mean Absolute Error) and MSE (Mean Squared Error) for regression problem, true positives, and negatives for predicting models, there is always a chance for your model to perform poorly when in the real world. Without a strategy to custom retrain automatically based on conditions it foresees in real-world scenarios, operationalizing AI projects are nightmares in disguise.
Enterprises investing in AI without a plan to solve the business problem are hit with increased cost. The inflation can be controlled by setting up proper computational resources and CI/CD pipelines for both model training and inference along with monitoring capabilities to alert on model degradation.