Homebreadcumb forward arrow iconBlogbreadcumb forward arrow iconArtificial Intelligencebreadcumb forward arrow iconAdversarial Machine Learning: Concepts, Types of Attacks, Strategies & Defenses

Adversarial Machine Learning: Concepts, Types of Attacks, Strategies & Defenses

Last updated:
2nd May, 2021
Read Time
9 Mins
share image icon
In this article
Chevron in toc
View All
Adversarial Machine Learning: Concepts, Types of Attacks, Strategies & Defenses

The exponential progress of the previous decades has propelled modern technological advancements in today’s world. We are currently a part of the ongoing ‘Industry 4.0’, at the centre of which are technologies like AI and ML. This industrial revolution involves a global transition towards scientific research and innovation in technologies of neural networks, Machine Learning, and Artificial Intelligence, IoT, digitisation, and much more.

Best Machine Learning and AI Courses Online

They provide us with an array of benefits in sectors like e-commerce, manufacturing, sustainability, supply chain management, etc. The global market for AI/ML is expected to surpass USD 266.92 billion by 2027 and continues to be a preferred choice of career for graduates everywhere.  

While the adaptation of these technologies is paving the way for the future, we are unprepared for events like Adversarial Machine Learning (AML) attacks. Machine Learning systems that are designed using coding languages like SML, OCaml, F#, etc., rely on programmable codes that are integrated throughout the system.

Ads of upGrad blog

External AML attacks performed by experienced hackers pose a threat to the integrity and accuracy of these ML systems. Slight modifications to the input data set can cause the ML algorithm to misclassify the feed, and thus reduce the reliability of these systems.

In-demand Machine Learning Skills

Learn Machine Learning  online from the World’s top Universities – Masters, Executive Post Graduate Programs, and Advanced Certificate Program in ML & AI to fast-track your career.

To equip yourself with the right resources for designing systems that can withstand such AML attacks, enrol in a PG Diploma in Machine Learning offered by upGrad and IIIT Bangalore.

Concepts Centred on Adversarial Machine Learning

Before we delve into the topic of AML, let us establish the definitions of some of the basic concepts of this domain:

  • Artificial Intelligence refers to the ability of a computing system to perform logic, planning, problem-solving, simulation, or other kinds of tasks. An AI mimics human intelligence due to the information fed into it by using Machine Learning techniques.
  • Machine Learning employs well-defined algorithms and statistical models for computer systems, which rely on performing tasks based on patterns and inferences. They are designed to execute these tasks without explicit instructions, and instead use predefined information from neural networks.
  • Neural Networks are inspired by the biological functioning of a brain’s neurons, which are used for systematically programming the observational data into a Deep Learning model. This programmed data helps decipher, distinguish, and process input data into coded information to facilitate Deep Learning.
  • Deep Learning uses multiple neural networks and ML techniques to process unstructured and raw input data into well-defined instructions. These instructions facilitate building multi-layered algorithms automatically through its representation/feature learning in an unsupervised manner.
  • Adversarial Machine Learning is a unique ML technique that supplies deceptive inputs to cause malfunction within a Machine Learning model. Adversarial Machine Learning exploits vulnerabilities within the test data of the intrinsic ML algorithms that make up a neural network. An AML attack can compromise resultant outcomes and pose a direct threat to the usefulness of the ML system.

To learn the key concepts of ML, such as Adversarial Machine Learning, in-depth, enrol for the Masters of Science (M.Sc) in Machine Learning & AI from upGrad.

Types of AML Attacks 

Adversarial Machine Learning attacks are categorised based on three types of methodologies.

They are:

1. Influence on Classifier

Machine Learning systems classify the input data based on a classifier. If an attacker can disrupt the classification phase by modifying the classifier itself, it can result in the ML system losing its credibility. Since these classifiers are integral to identifying data, tampering with the classification mechanism can reveal vulnerabilities that can be exploited by AMLs.

2. Security Violation

During the learning stages of an ML system, the programmer defines the data that is to be considered legitimate. If legitimate input data is improperly identified as malicious, or if malicious data is provided as input data during an AML attack, the rejection can be termed as a security violation.

3. Specificity

While specific targeted attacks allow specific intrusions/disruptions, indiscriminate attacks add to the randomness within the input data and create disruptions through decreased performance/failure to classify.

AML attacks and their categories are conceptually branched out of the Machine Learning domain. Due to the rising demand for ML systems, nearly 2.3 million job vacancies are available for ML and AI engineers, according to Gartner.[2]  You can read more about how Machine Learning Engineering can be a rewarding career in 2021.

Adversarial Machine Learning Strategies

To further define the goal of the adversary, their prior knowledge of the system to be attacked and the level of possible manipulation of data components can assist in defining Adversarial Machine Learning strategies.

They are: 

1. Evasion

ML algorithms identify and sort the input data set based on certain predefined conditions and calculated parameters. The evasion type of AML attack tends to evade these parameters used by algorithms to detect an attack. This is carried out by modifying the samples in a manner that can avoid detection and misclassify them as legitimate input.

They do not modify the algorithm but instead spoof the input by various methods so that it escapes the detection mechanism. For example, anti-spam filters that analyse the text of an email are evaded with the use of images that have embedded text of malware code/links. 

2. Model extraction

Also known as ‘model stealing’; this type of AML attacks is carried out on ML systems to extract the initial training data used for building the system. These attacks are essentially capable of reconstructing the model of that Machine Learning system, which can compromise its efficacy. If the system holds confidential data, or if the nature of the ML itself is proprietary/sensitive, then the attacker could use it for their benefit or disrupt it.

3. Poisoning

This type of Adversarial Machine Learning attack involves disruption of the training data. Since ML systems are retrained using data collected during their operations, any contamination caused by injecting samples of malicious data can facilitate an AML attack. For poisoning data, an attacker needs access to the source code of that ML and retrains it to accept incorrect data, thus inhibiting the functioning of the system.

Proper knowledge of these Adversarial Machine Learning attack strategies can enable a programmer to avoid such attacks during operation. If you need hands-on training for designing ML systems that can withstand AML attacks, enrol for the Master’s in Machine Learning and AI offered by upGrad.

Specific Attack Types

Specific attack types that can target Deep Learning systems, along with conventional ML systems like linear regression and ‘support-vector machines’, can threaten the integrity of these systems. They are: 

  • Adversarial examples, such as FMCG, PGD, C&W, and patch attacks, cause the machine to misclassify, as they appear normal to the user. Specific ‘noise’ is used within the attack code to cause malfunction of the classifiers.
  • Backdoor/Trojan attacks overload an ML system by bombarding it with irrelevant and self-replicating data that prevents it from optimum functioning. These Adversarial Machine Learning attacks are difficult to protect from, as they exploit the loopholes that exist within the machine.
  • Model Inversion rewrites classifiers to function in an opposite manner to which they were originally intended. This inversion prevents the machine from performing its basic tasks due to the changes applied to its inherent learning model.
  • Membership Inference Attacks (MIAs) can be applied to SL (supervised learning) and GANs (Generative Adversarial Networks). These attacks rely on the differences between the data sets of initial training data and external samples that pose a privacy threat. With access to the black-box and its data record, inference models can predict whether the sample was present in the training input or not.

To protect ML systems from these types of attacks, ML programmers and engineers are employed across all the major MNCs. Indian MNCs that host their R&D centres to encourage innovation in Machine Learning, offer salaries ranging from 15 to 20 Lakh INR per annum.[3] To learn more about this domain and secure a hefty salary as an ML engineer, enrol in an Advanced Certification in Machine Learning and Cloud hosted by upGrad and IIT Madras.

Defences Against AMLs

To defend against such Adversarial Machine Learning attacks, experts suggest that programmers rely on a multi-step approach. These steps would serve as countermeasures to the conventional AML attacks described above. These steps are:

  • Simulation: Simulating attacks according to the possible attack strategies of the attacker can reveal loopholes. Identifying them through these simulations can prevent AML attacks from having an impact on the system.
  • Modelling: Estimating the capabilities and potential goals of attackers can provide an opportunity to prevent AML attacks. This is done by creating different models of the same ML system that can withstand these attacks. 
  • Impact evaluation: This type of defence evaluates the total impact an attacker can have over the system, thus ensuring preparation in the event of such an attack.
  • Information laundering: By modifying the information extracted by the attacker, this type of defence can render the attack pointless. When the extracted model contains purposely placed discrepancies, the attacker cannot recreate the stolen model.

Examples of AMLs

Various domains within our modern technologies are directly under the threat of Adversarial Machine Learning attacks. Since these technologies rely on pre-programmed ML systems, they could be exploited by people with malicious intentions. Some of the typical examples of AML attacks include:

1. Spam filtering: By purposely misspelt ‘bad’ words that identify spam or the addition of ‘good’ words that prevent identification.

2. Computer security: By hiding malware code within cookie data or mislead digital signatures to bypass security checks.

3. Biometrics: By faking biometric traits that are converted to digital information for identification purposes.


Ads of upGrad blog

Popular AI and ML Blogs & Free Courses


As the fields of Machine Learning and Artificial Intelligence continue to expand, their applications increase across sectors like automation, neural networks, and data security. Adversarial Machine Learning will always be significant for the ethical purpose of protecting ML systems and preserving their integrity. 

If you are interested to know more about machine learning, check out our Executive PG Programme in Machine Learning and AI program which is designed for working professionals and provide 30+ case studies & assignments, 25+ industry mentorship sessions, 5+ practical hands-on capstone projects, more than 450 hours of rigorous training & job placement assistance with top firms.


Pavan Vadapalli

Blog Author
Director of Engineering @ upGrad. Motivated to leverage technology to solve problems. Seasoned leader for startups and fast moving orgs. Working on solving problems of scale and long term technology strategy.
Get Free Consultation

Select Coursecaret down icon
Selectcaret down icon
By clicking 'Submit' you Agree to  
UpGrad's Terms & Conditions

Our Popular Machine Learning Course

Frequently Asked Questions (FAQs)

1Are Adversarial attacks malignant to cybersecurity?

Cybersecurity is a priority in the digital era. It is also fragile as it stands to withhold threats to its structure and function. If not protected, the IoT becomes vulnerable to theft of privacy, damage, and misuse. Adversarial attacks can occur when the parameters set by algorithms are evaded by misclassifying the incoming attack as input data. This is just one way of breaching machine learning systems. With an increasing number of adversarial attacks on machine learning algorithms, the safety, and efficacy of cybersecurity are compromised. Adversarial Machine Learning has been developed to tackle these threats.

2How are Machine Learning Systems vulnerable?

Machine Learning Systems have to be robust in their structure to be reliable. There has been a swarm of malicious attacks on these systems in recent years, adversarial attacks breach the protective barriers of these systems. This happens through duplicating the model and changing the initial training data, deceiving the algorithm’s parameters, or retraining existing data to overlook incoming attacks. These adversarial attacks can breach ethical parameters and manipulate systems to operate unauthorised functions. Adversarial Machine Learning defences are used to identify invading malware attacks and restore the integrity of the system.

3What strategies work for combating Adversarial attacks?

Adversarial attacks can range from infringement of data to manipulation of the entire system. They are pervasive and can spread across the system fast if the attacker has strong access to the system’s algorithm. An example is spam filtering, wherein the words are structured so that the algorithm cannot detect them as spam. Using Adversarial Machine Learning, programmers make measures against these attacks. They try to simulate attacks by visualising the attacker’s strategy, recognizing their pattern of invasion, and identifying loopholes. They use these insights to protect the system from further breaches. Testing the scope of the attacker’s skills and capability can help in modifying and safeguarding the system.

Explore Free Courses

Suggested Blogs

Top 9 Python Libraries for Machine Learning in 2024
Machine learning is the most algorithm-intense field in computer science. Gone are those days when people had to code all algorithms for machine learn
Read More

by upGrad

19 Feb 2024

Top 15 IoT Interview Questions & Answers 2024 – For Beginners & Experienced
These days, the minute you indulge in any technology-oriented discussion, interview questions on cloud computing come up in some form or the other. Th
Read More

by Kechit Goyal

19 Feb 2024

Data Preprocessing in Machine Learning: 7 Easy Steps To Follow
Summary: In this article, you will learn about data preprocessing in Machine Learning: 7 easy steps to follow. Acquire the dataset Import all the cr
Read More

by Kechit Goyal

18 Feb 2024

Artificial Intelligence Salary in India [For Beginners & Experienced] in 2024
Artificial Intelligence (AI) has been one of the hottest buzzwords in the tech sphere for quite some time now. As Data Science is advancing, both AI a
Read More

by upGrad

18 Feb 2024

24 Exciting IoT Project Ideas & Topics For Beginners 2024 [Latest]
Summary: In this article, you will learn the 24 Exciting IoT Project Ideas & Topics. Take a glimpse at the project ideas listed below. Smart Agr
Read More

by Kechit Goyal

18 Feb 2024

Natural Language Processing (NLP) Projects & Topics For Beginners [2023]
What are Natural Language Processing Projects? NLP project ideas advanced encompass various applications and research areas that leverage computation
Read More

by Pavan Vadapalli

17 Feb 2024

45+ Interesting Machine Learning Project Ideas For Beginners [2024]
Summary: In this Article, you will learn Stock Prices Predictor Sports Predictor Develop A Sentiment Analyzer Enhance Healthcare Prepare ML Algorith
Read More

by Jaideep Khare

16 Feb 2024

AWS Salary in India in 2023 [For Freshers & Experienced]
Summary: In this article, you will learn about AWS Salary in India For Freshers & Experienced. AWS Salary in India INR 6,07,000 per annum AW
Read More

by Pavan Vadapalli

15 Feb 2024

Top 8 Exciting AWS Projects & Ideas For Beginners [2023]
AWS Projects & Topics Looking for AWS project ideas? Then you’ve come to the right place because, in this article, we’ve shared multiple AWS proj
Read More

by Pavan Vadapalli

13 Feb 2024

Schedule 1:1 free counsellingTalk to Career Expert
footer sticky close icon