We at upGrad have recently launched an Advanced Certification Program in Machine Learning & Cloud with IIT Madras. I have received a lot of queries from prospective learners about why we chose to teach these two skills and the below article is an attempt to explain the power of Machine Learning in the Cloud.
Two Primary Barriers:
There are two main roadblocks for the widespread application of machine learning. One, competence and the second, cost. And herein lies the power of machine learning in the cloud.
Most references to machine learning involve the Netflix recommendation engine or the Uber autonomous car or some other grand project. But for me, one of the most inspiring applications of machine learning is the less heard story of the Japanese farmer who used deep learning & TensorFlow to sort his cucumbers!
He used machine learning to save a significant amount of manual effort without any prior knowledge of the subject and with very limited investment. This is an example of the true democratization of machine learning and the potential it has to improve the status quo, for everybody.
First, let us address the question of competence. We have all heard of the “lack of talent” in the area of machine learning. While we continue to train more people to become machine learning experts, it is also equally important to simplify the process of applying machine learning. Cloud service providers like Amazon, Google & Microsoft have set up powerful systems to help build, train & deploy models with relative ease even if you do not have any expertise in the area.
Pre-existing libraries can now be deployed for data processing, model building/ training/ evaluation/ deployment, leading to accurate predictions & recommendations. This greatly reduces the requirement for millions of machine learning experts to drive adoption & impact.
Second, the question of cost. Deploying machine learning algorithms requires a lot of computing power and hence a large scale hardware infrastructure. Let us take the example of our Japanese farmer, who ran the neural network models on his Windows PC.
Even after converting the images to low resolution, it would take up to 3 days to train the model with 7000 images! Using a larger number of high-resolution pictures would significantly improve the accuracy, but would also drastically increase the training time with the computing power of a Windows PC.
In more advanced settings with real-time training/ prediction and fluctuating loads, the computing power requirement is very high and costly. This issue can be addressed by using low-cost cloud platforms for training/ prediction that dedicates hundreds of cloud servers to training a network via large scale distributed training. In this model that is now fairly standard, you can avoid large upfront capex investment, have flexible computing capacity and only pay for what you use.
Cloud service providers have basically reduced the entry barrier for machine learning by reducing the level of competence & cost required to use it effectively. For all applications like autonomous cars, IoT, smart connected homes and even for cucumber sorting, understating how to use the cloud infrastructure to effectively develop, train & deploy machine learning models is an important skill to master.
If you are interested to learn about cloud computing and Machine learning, upGrad in collaboration with IIT- Madras, has launched the Machine Learning in Cloud program. The course will equip you with the necessary skills for this role: maths, data wrangling, statistics, programming, cloud-related skills, as well as ready you for getting the job of your dreams.