AI and ML algorithms are being integrated in various (malicious) applications

Abusing AI Cloud Services

It is important to note that ML algorithms do not necessarily need to run on the same host machine where

the malware runs. Another approach to detect if malicious actors have already started leveraging ML

when developing malware is to check, therefore, whether a particular malware connects to cloud-based

AI services.

To follow this approach, we compiled a list of all AI services offered by major cloud providers, including

Amazon AWS, Microsoft Azure, Google Cloud, Alibaba Cloud, Yandex, and others. All of these have

started offering AI services in recent years: For example, Amazon, through its AWS platform, offers

image recognition services (Amazon Rekognition), unstructured text analytics (Amazon Comprehend), or

named entity extraction (Amazon Textract), with competitors offering similar services of their own. Each

of these services can be accessed via an HTTPS connection to a specific endpoint URL, which uses a

specific pattern. For example, all services from Amazon’s AWS platform contain “aws.amazon.com” in

the hostname and the name of the service in the path.

Using those patterns, we performed an extensive search within Trend Micro’s databases, focusing on two

datasets in particular:

• The first dataset contains behavioral information of malware samples that have been analyzed in the

past.

• The second dataset contains anonymized reports of connections performed by malicious software as

detected by Trend Micro’s antivirus software.

By using these patterns and focusing on the two datasets, it was possible to check if any novel malware

samples found on the internet connects to AI cloud services and why, or if any malicious connection to

said services that come from a malicious software on a victim’s machine had been detected previously.

As of writing this report, our search did not identify any traces of malicious software exploiting cloud

services. There are two possible reasons for this lack of identification. First, scaling matters (from the

point of view of the malicious actor) might result in higher costs. Second, without proper measures, the

move to exploit cloud services might increase chances of the malicious actor being revealed.

Nevertheless, cloud services should be monitored for this kind of connection, since the next iteration of

AI malware targeting them might still emerge.

Abusing Smart Assistants

An alternative approach to attacking AI algorithms is to target AI assistants by either exploiting their

presence in households or abusing their development model. In particular, an AI assistant could be

targeted by developing adversarial systems or polluting datasets.

An example of an adversarial system is elaborated on in the Trend Micro paper “The Sound of a Targeted

Attack,”26 where an attacker could exploit exposed smart speakers to issue audio commands to a nearby

smart assistant, such as Amazon Alexa or Google Home. If a speaker is connected to the internet, its

vulnerabilities could be exploited, causing it to play an audio file hosted on an arbitrary web address

set up by the attacker. It is also possible for the file to contain speech that issues a specific command

to the nearby smart assistants. Additionally, a stealth attack might use an issued command that is not

perceivable by the human ear.27 Exposed devices can be easily found using services such as Shodan, a

search engine for internet-connected devices and systems.

The possibility of such attacks is further exacerbated by the fact that smart assistants are often in control

of home automation systems. In “Cybersecurity Risks in Complex IoT Environments: Threats to Smart

Homes, Buildings and Other Structures,”28 Trend Micro proves that hijacking a smart assistant through

exposed audio devices is only one of the many attacks that can be carried out by a malicious actor

interested in breaking into a smart home.

AI-Supported Password Guessing

Another application involves employing ML to improve password-guessing algorithms. Despite the fact

that this application is being researched with moderate success, it is important to note that this same

application has already proven to be more efficient than more traditional approaches. It is also quite hard

to detect on its own. As a result, it is not far-fetched to presume that AI-supported algorithms and tools

are in constant conceptualization and development by individuals or groups who might abuse these.

Traditional password-guessing tools, such as HashCat29 and John the Ripper,30 usually work by comparing

many different variations to the password hash in order to identify the password that corresponds to

the hash. Attempts are generated from a dictionary of frequently used passwords; after, variations are

made based on the composition of the password. For example, variations of “password” might be

“password12345” or “p4ssw0rd.”

Through the use of neural networks and generative adversarial networks (GANs) in particular, it is possible to analyze a large dataset of passwords and generate variations that fit the statistical distribution, such as for password leaks. This leads to more targeted and more effective password guesses.

An early attempt at this is already evident in a post on an underground forum from February 2020. The

post mentions a GitHub repository from the same year, in which a software is able to parse through 1.4

billion credentials and generate password variation rules based on its findings.

AI-Supported CAPTCHA Breaking

The application of ML for breaking CAPTCHA security systems is frequently addressed on criminal forums.

CAPTCHA images are commonly used on websites to thwart criminals when they attempt to abuse web

services — particularly when they try to automate attacks (some attempts involve creating new accounts

or adding new comments or replies on forums, among others). Developing systems that try to break

CAPTCHA images to automate the abuse of those services would thus be a natural progression for

cybercriminals.

Software that implements neural networks to solve CAPTCHAs, such as XEvil 4.0,33 is currently being

tested on criminal forums. Moreover, it has been claimed that this neural network can break humanrecognition

systems using CAPTCHA on Yandex pages. The tool utilizes 80 to 100 CPU threads (with the

CPUs running in parallel) to speed up CAPTCHA solving. It is also advertised on Russian underground

forums and rented out to users for 4,000 rubles weekly (approximately US$54 as of writing) or 10,000

rubles monthly (approximately US$136 as of writing).

In order to make it more difficult to break CAPTCHA and diminish the efficiency of AI algorithms, CAPTCHA

developers should increase the variety of color patterns and shapes. However, given the way that the anti-

CAPTCHA defense engines work, it is possible that doing so would cause only a very slight delay in the

time it takes for criminals to break the CAPTCHAs. Barring that, perhaps it would be worth developing

some sort of software filter to detect scripts or programs that repeatedly try to solve CAPTCHAs too

quickly. This software filter could also be developed to send completely fake, unsolvable images to these

scripts or programs in order to try and poison their datasets.