AI has been the buzz for several years now. When OpenAI unleashed ChatGPT on the public in late 2022, it went from being a murmur to an all-out roar. Since then all of the major tech companies like Microsoft, Google, Meta, and Apple have released their own flavor of AI. There has been a lot of discourse about whether AI should be regulated or even that AI will be the end of us all.

Let's put all of that aside for a minute and look at the some of the power that AI holds and how it can be used for both good and bad. Like all technology, AI can be abused and used for nefarious purposes. When it comes to cybersecurity this is even more true.

On the positive side AI is being used to improve the overall security of software by analyzing code and looking for weaknesses. The can help developers correct problems within their product. AI is increasingly being used to originate codes as well. This can lead to more secure code in some cases. However at the recent Infosecurity Europe 2024 Conference Lucas von Stockhausen Lucas, executive director for application security engineering at Synopsys, discussed the fact that large language model (LLM)-based tools have a tendency to reuse vulnerable or improperly written code, just as they sometimes provide incorrect information or hallucinations. Meanwhile bad actors are using AI to find weaknesses to exploit within software. With so much software out there today based on open source code this is not a difficult task. It is a game of cat and mouse.

Security vendors are now offering penetration testing products that utilize AI to find weaknesses in organizations' defenses. These tools identify the vulnerabilities and allow the organization to remediate. However, these tools can be used by bad actors as well to spot the weaknesses and allow then to penetrate the organizations perimeter.

On the business side of AI technology, new products like Copilot for Microsoft 365 provide end users with incredibly powerful analysis capabilities of the data to which they have access. Within MS Office apps users ask copilot to summarize the data within an excel spreadsheet, compose an email pulling information from Outlook, Teams, and Word, or build a PowerPoint presentation for a certain topic. This is an incredibly powerful productivity tool. However, this tool will access all of the information the user is given access to, whether they know it or not. We've all been there where someone in the organization saves confidential information to a shared drive inadvertently. In the past another user had to stumble upon that document in order for it to be discovered. Now Copilot will find it if the content is relevant to the request of the user. For example, if the user is looking for general information about the average salaries within an industry, Copilot may find a spreadsheet with the salaries of all employees that was placed in a public location by a payroll worker. This information is now exposed to all.

 Copilot as a Phishing Tool

Now let's look at combining the power of Copilot with a Business Email Compromise (BEC). The bad actor gains control of the email account of the CEO. Once in the account as the user they now have all of the access the actual user has. Using Copilot the hacker queries the email and other files for key information such as other businesses with which the company processes financial transactions. From this they obtain an easy target to request a phony bank transfer. They can use Copilot to craft an email in the style of writing the CEO would typically use. The request is sent from the CEOs account to the CFO requesting the wire transfer. To the CFO the email looks legit as it contains the proper details about the company and looks like any other email crafted by the CEO. Copilot has done its job.

Another technology developed by Microsoft is VALL-E. VALL-E is a voice synthesis technology that claims to be able to create a very close representation of a speakers voice that can then be used to recite any typed text. This is nothing new, however, Microsoft is able to do this with only a 3-second sample of the speaker's voice. This is not available to the public and Microsoft has no plans to incorporate the technology into any products…yet. The fact that there are other commercial products already on the market that can perform similarly with 10-15 second samples would suggest it is only a matter of time.

Now let's take our BEC example from above and amp it up a notch. Using voice cloning technology the hacker calls the CEO with the intent to record a sample of their voice. Even if they only get their out-of-office voicemail message, they can still capture enough of a sample to create a reasonable clone of the CEOs voice. Now to add even more credibility to the wire transfer request, the hacker sends a recorded voicemail with the CEOs cloned voice to the CFO saying, "Hey, I just sent you an email with the details of a wire transfer, I just wanted to make sure you got it because it needs to be done within 30 minutes. I'm getting on a plane so you won't to reach me for a couple of hours".  This is now a very believable request and the CFO would likely be duped by it.

If Star Wars has taught us anything, there is a dark side to any power. As Artificial Intelligence evolves it will offer incredible gains in productivity and take the drudgery out of many tasks. However, we must protect ourselves against those who would use it against us.

The basics still apply. We must make sure that we are practicing even the most basic cyber hygiene:

• In our BEC example above the use of strong passwords and multifactor authentication by the CEO would make that exploit much more difficult.

• When deploying Microsoft Copilot for MS365 make sure that user permissions are accurate and sensitive data is not carelessly placed in public areas 

• IT departments and MSPs should be using SEIM and SOC solutions (AI powered!) to identify anomalies and strange behavior within their networks. You'll need to use AI as a defense to defeat AI as a weapon. 

OTX Roundtable GRC

This is an example of some of the topics we tackle within OTX Roundtable GRC. We look at emerging technology from both a positive and a problematic standpoint. We discuss other topics such as contractual integrity, cyber insurance, and compliance with cybersecurity frameworks. Members set goals and hold each other to achieving them. Successes, challenges, and lessons learned are shared among members. Together, members improve their MSP practices, gain a competitive edge, and command higher rates.

If you would like to find out more about joining OTX Roundtable GRC or find out more about other services OTX Partners offers please click here.