'Adversarial machine learning' & other potential AI risks
Growing use of artificial intelligence and machine learning is opening up new lanes of vulnerability.
Accompanying the growing adoption of artificial intelligence at organizations of all sizes are growing risks posed by adversarial machine learning (ML), according to Swiss Re’s latest SONAR report.
Adversarial ML, according to Swiss Re, is a “threat that encompasses any targeted exploitation or hacking of AI systems, leveraging ML-specific vulnerabilities.”
These vulnerabilities are presenting new risks for insurers and policyholders, Swiss Re noted, adding in the report that hackers can trick ML models into making mistakes or leaking information. ML Models can have their training data corrupted.
In addition to cyber insurance, adversarial ML and the failures that follow an incident could trigger claims in professional indemnity and E&O lines. Further, the theft of an ML model could result in the loss of intellectual properties, and in a worst-case scenario, they could cause accidents that led to casualty claims.
3 potential vulnerabilities
As AI and machine learning, which is a subset of artificial intelligence, continue to see wider and wider uses, the diversity of potential vulnerabilities grows.
The following are three potential examples of what can happen when AI gets hacked, per Swiss Re:
1. Backdoor data poisoning
ML models are designed to perform well-defined tasks, like detecting spam or classifying images. However, nefarious actors can bake “backdoors,” or secret behaviors that are triggered by specially crafted inputs, into models. A common ML backdoor method is data poisoning, which involves targeting a model’s training data to implant a trigger mechanism.
According to Swiss Re, targeted data poisoning could result in high failure rates in autonomous cars, for example, in turn causing casualty claims.
Further, bad actors can plant backdoors that are undetectable by “any computationally-bound observer,” according to a paper from UC Berkeley, MIT, and the Institute for Advanced Study researchers.
2. Model evasion: Does this look right to you?
Unlike data poisoning, model evasion doesn’t aim to alter the ML system, but rather exploits its weaknesses to cause errors.
This is typically done by slightly altering an input so it is misclassified by the system, but undetectable to a person. For example, a sticker of a visual pattern could be applied to a car that tricks an automated claims tool into misjudging damage, Swiss Re reported.
3. Leaking training data
In addition to attacks on the ML models themselves, hackers could also target these systems for the data they are trained on, which can often include confidential information, Swiss Re pointed out.
Attackers could run a series of targeted queries to extract this training data, raising concerns over data protection.
To mitigate these ML risks, Swiss Re said strict access management, usage limits and data governance are critical. Additionally, not exposing the models to the internet and using only trusted data are power strategies, although not always possible.
Related: