Is AI Actually Dangerous?

There’s been a lot of noise recently about whether or not AI is dangerous. However, the conversation is missing a key distinction about what actually makes something dangerous. While the frontier labs want you to believe their tools are dangerous from how capable they are, I believe the real danger comes from their recklessness and incompetence in how these AI tools are being developed.

Defining “Dangerous”

Before diving in, I want to make a distinction about two different forms of “dangerous”. The more common form of being dangerous is when something is intentionally dangerous. This is when someone or something is highly skilled and has malicious intent. Let’s call this Type I dangerous. An example of this would be an elite military unit, like the Navy Seals. The entity knows what it’s doing and is intentionally seeking to cause damage or disruption. The second kind of dangerous is being dangerous through incompetence. Someone/something that behaves recklessly or impertinently in such a manner that they do damage either to themselves or those around them. Chernobyl is probably the most infamous example of this. In this instance, the USSR was dangerous to itself through its own incompetence. Let’s call this Type II dangerous.

How the Frontier Labs Misconstrue Danger

Getting back to how this applies to AI, the recent headlines have all tried to paint AI as Type I dangerous. The frontier labs CEOs have all been expressing concerns about how the technology they are developing could be harmful to humanity, even possibly leading to our extinction. Additionally, engineers at these companies have raised their own concerns about how dangerous this technology is and/or will soon become. The assumption behind these claims is that AI will become so advanced and capable that it will be able to inflict real harm to humans. This is certainly how I believe many people interpret these comments from those in the industry. But I’ve used AI extensively the past two year in my role as a DevOps engineer, and I have a hard time seeing this come to fruition. I’ve worked with it basically every day for the last two years and it’s sometimes shocking just how bad all of the LLMs are at interpreting context, often making very trivial and obvious mistakes. This is because, at their cores, these systems don’t actually understand anything. They are stochastic models that pick the next “token” or word, based on what came before and the corpus of information the model was trained on. However, this is not to say that I believe AI is entirely safe. While I believe that a malevolent and capable AI is not likely (despite what the frontier labs would like you to think), I believe AI can become dangerous simply through how incompetent it is - aka Type II dangerous.

I believe we’ve already seen this in action. One of the most talked about events recently was the OpenAI “hack” of Hugging Face. OpenAI was very quick to position this as a sign that their agents were extremely capable. But I would argue this hack was only possible through incompetence, arrogance, and recklessness. First, OpenAI claimed these agents were running in a Sandbox, i.e. an isolated and locked down environment. No matter how “advanced” anyone/anything is, you should not be able to escape a proper sandbox. OpenAI should be taking a real hard look at their development practices and hire some better DevOps engineers. A true sandbox is inescapable, they should be focusing on building better sandboxes rather than defaulting to thinking that their agents are “super-human”. Secondly, the manner in which these frontier labs are developing their agents is completely reckless. The recent trend in the industry is using a loop. This means that when an agent is kicked off, it generates a response to the initial input, and then uses the output as the input to the next loop of the agent. This is done over and over and over again either a set number of times, until a result is achieved, or until something breaks. The issue here, is that the output generated by AI is non-deterministic. Thus over the course of thousands of loops, the agent could end up working on something completely beyond or outside of the initial request made by the end-user. This is a dangerous and unethical way of doing development, especially when the sandboxes where these agents are running are so poorly developed and easily escapable. Lastly, and most egregiously, OpenAI admitted that when they were running these tests they had removed many of the safeguards that exist when typical users are using the model. To remove these safeguards, while simultaneously running the agent in a loop architecture, all while running in a poorly architected sandbox…it’s not hard to see that this experiment was always doomed to fail.

What Do We Do About It?

So, is AI dangerous? I believe the answer is yes, but not in the way the frontier labs want you to think it is. It’s clear, that at the present moment, AI is dangerous due to the manner in which it is being developed. In the race to produce the most advanced AI, the frontier labs have eschewed many proper development practices and taken several shortcuts which have left them with a non-deterministic tool which they are unleashing onto the internet with little to no safeguards. But it doesn’t have to be this way. Maybe instead of listening to the CEOs of these companies, we can begin demanding they begin developing these tools in a more ethical manner, and hold them accountable when they fall short of that standard. Technology should be built in a way that it augments the human experience, not endanger it, whether intentionally or not.