Navigating generative AI risks and regulatory challenges

The mass availability of generative AI, such as OpenAI’s ChatGPT and Google Bard, became a top concern for enterprise risk executives in the second quarter of 2023, according to Gartner.

A benchmarked view of emerging risks

“Generative AI was the second most-frequently named risk in our second quarter survey, appearing in the top 10 for the first time,” said Ran Xu director, research in the Gartner Risk & Audit Practice. “This reflects both the rapid growth of public awareness and usage of generative AI tools, as well as the breadth of potential use cases, and therefore potential risks, that these tools engender.”

In May 2023, Gartner surveyed 249 senior enterprise risk executives to provide leaders with a benchmarked view of 20 emerging risks. The report includes detailed information on the possible impact, time frame, level of attention, and perceived opportunities for these risks.

Third-party viability was the top fast-emerging risk that organizations are monitoring most closely in the 2Q23 survey. Financial planning uncertainty was the third ranked risk, followed by cloud concentration risk. China trade tensions rounded out the top five risks that were split between issues symptomatic of the current broad macroeconomic and geopolitical volatility, and technology-related concerns.

Generative AI availability

Gartner has previously identified six risks of generative AI and four areas of AI regulation that are relevant to assurance functions. In terms of managing enterprise risk, three main aspects must be addressed, according to Gartner experts:

Intellectual property

“Information entered into a generative AI tool can become part of its training set, meaning that sensitive or confidential information could end up in outputs for other users,” said Xu. “Using outputs from these tools could well end up inadvertently infringing the intellectual property rights of others who have used it.”

It’s important to educate corporate leadership on the necessity for caution and transparency around the use of such tools so that intellectual property risks can be properly mitigated both in terms of input and output from generative AI tools.

Data privacy

Generative AI tools may possibly share user information with third parties, such as vendors or service providers, without prior notice. This has the potential to violate privacy law in many jurisdictions. For example, regulation has already been implemented in China and the EU, with proposed regulations emerging in USA, Canada, India and UK among others.

Cybersecurity

“Hackers are always testing new technologies for ways to subvert it for their own ends, and generative AI is no different,” said Xu. “We’ve seen examples of malware and ransomware code that generative AI has been tricked into producing, as well as ‘prompt injections’ attacks that can trick these tools into giving away information they should not. This is leading to the industrialization of advanced phishing attacks.”

Causes and implications of third-party viability risk

“Persistent inflation that is less responsive to interest rate rises and contuse longer than anticipated has escalated costs and margin pressures on third parties,” said Xu. “As central banks increase interest rates to fight inflation, this also brings about a process of credit tightening that may force suppliers to suspend operations or become insolvent as borrowing costs rise.”

If economic conditions deteriorate broadly, this may cause an unexpected drop in demand that could affect vendor viability or their ability to provide goods and services in a timely manner.

Gartner experts identified three potential third-party viability consequences for risk managers to monitor as the situation develops:

• Loss of key inputs and materials: If third-parties are increasing their prices due to the wider economic situation there is a clear risk of losing access to key inputs and materials as third parties would favour customers willing to pay higher prices.
• Flawed financial planning assumptions: Cost assumptions will be rendered invalid as suppliers increase prices or fail, necessitating switching costs and increased prices for obtaining goods and services.
• Challenges outside the supply chain: Partners, such as managed service providers or commercial partners, creditors, or technology vendors may cease or curtail operations.