Is it time to ‘shield’ AI with a firewall? Arthur AI thinks so

With the risks of hallucinations, private data information leakage and regulatory compliance that face AI, there is a growing chorus of experts and vendors saying there is a clear need for some kind of protection.

One such organization that is now building technology to protect against AI data risks is New York City based Arthur AI. The company, founded in 2018, has raised over $60 million to date, largely to fund machine learning monitoring and observability technology. Among the companies that Arthur AI claims as customers are three of the top-five U.S. banks, Humana, John Deere and the U.S. Department of Defense (DoD).

Arthur AI takes its name as an homage to Arthur Samuel, who is largely credited for coining the term “machine learning” in 1959 and helping to develop some of the earliest models on record. 

Arthur AI is now taking its AI observability a step further with the launch today of Arthur Shield, which is essentially a firewall for AI data. With Arthur Shield, organizations can deploy a firewall that sits in front of large language models (LLMs) to check data going both in and out for potential risks and policy violations.

“There’s a number of attack vectors and potential problems like data leakage that are huge issues and  blockers to actually deploying LLMs,” Adam Wenchel, the cofounder and CEO of Arthur AI, told VentureBeat. “We have customers who are basically falling all over themselves to deploy LLMs, but they’re stuck right now and they’re using this they’re going to be using this product to get unstuck.”

Do organizations need AI guardrails or an AI firewall?

The challenge of providing some form of protection against potentially risky output from generative AI is one that multiple vendors are trying to solve.

>>Follow VentureBeat’s ongoing generative AI coverage<<

Nvidia recently announced its NeMo Guardrails technology, which provides a policy language to help protect LLMs from leaking sensitive data or hallucinating incorrect responses. Wenchel commented that from his perspective, while guardrails are interesting, they tend to be more focused on developers.

In contrast, he said where Arthur AI is aiming to differentiate with Arthur Shield is by specifically providing a tool designed for organizations to help prevent real-world attacks. The technology also benefits from observability that comes from Arthur’s ML monitoring platform, to help provide a continuous feedback loop to improve the efficacy of the firewall.

How Arthur Shield works to minimize LLM risks

In the networking world, a firewall is a tried-and-true technology, filtering data packets in and out of a network.

It’s the same basic approach that Arthur Shield is taking, except with prompts coming into an LLM, and data coming out. Wenchel noted some prompts that are used with LLMs today can be fairly complicated. Prompts can include user and database inputs, as well as sideloading embeddings.

“So you’re taking all this different data, chaining it together, feeding it into the LLM prompt, and then getting a response,” Wenchel said. “Along with that, there’s a number of areas where you can get the model to make stuff up and hallucinate and if you maliciously construct a prompt, you can get it to return very sensitive data.”

Arthur Shield provides a set of prebuilt filters that are continuously learning and can also be customized. Those filters are designed to block known risks — such as potentially sensitive or toxic data — from being input into or output from an LLM.

“We have a great research department and they’ve really done some pioneering work in terms of applying LLMs to evaluate the output of LLMs,” Wenchel said. “If you’re upping the sophistication of the core system, then you need to upgrade the sophistication of the monitoring that goes with it.”