AI agents are becoming capable of doing much more than answering questions or generating content. They can work through tasks, interact with systems and keep trying to achieve an objective with limited human intervention.

That is creating a new cybersecurity challenge: if an AI agent can act on its own, who is responsible for securing it?

For Nadav Zafrir, CEO of Check Point, the recent incident involving an AI model and Hugging Face was a warning about where the technology is heading.

“The box is open. Is this a sort of a Frankenstein moment? It is,” Zafrir told CNBC-TV18.

But his bigger concern is not just about one incident. It is about how companies should build security around AI as these systems become more autonomous.

Why an AI agent cannot secure another AI agent

Traditional cybersecurity has largely focused on protecting networks, software, devices and users. AI agents change the equation because the software itself can increasingly take actions and pursue an assigned objective.

That makes the security layer around an AI system particularly important.

Zafrir’s message is blunt: “You cannot allow the agent to secure the agent.”

In other words, the system performing a task should not be the only system deciding whether its own actions are safe. Security controls need to sit outside the model.

Zafrir said Check Point is training its own model to act as a kind of guardian. While that model may be “really lousy at writing poetry”, he said it is being continuously trained on the latest attacks so that it can protect the AI system from the outside.

The idea is to have an independent layer that can monitor what an AI agent is doing rather than relying entirely on the agent itself to recognise when it has crossed a security boundary.

That becomes more important as AI systems are given access to more data, applications and external systems.

AI is capable, but it is not human

There is another problem, Zafrir argues: people tend to think of AI models and agents as if they were highly intelligent humans.

They are not.

AI systems can be extremely capable and persistent. They can keep working towards an objective and do not get tired or lazy in the way humans do. But they do not have the same context, judgement or morality that people bring to a decision.

That distinction is important.

An AI agent may be very good at completing an assignment, but it does not necessarily understand whether the objective itself makes sense, whether a particular action is appropriate or what its consequences might be.

Zafrir’s warning is therefore less about treating AI as malicious and more about understanding its limitations.

“We need to be humble about it,” he said, arguing that people should stop thinking of AI systems as intelligent human beings and instead recognise them for what they are: highly capable algorithms with access to vast amounts of information.

That humility, he said, is also important because “we still don’t understand everything.”

Cyberattacks are being industrialised

The threat is not limited to autonomous AI agents. Zafrir also sees a broader change in the cyber threat landscape.

Capabilities that were once available only to a small number of sophisticated attackers are becoming accessible to many more people.

But Zafrir believes the word “democratisation” does not fully describe what is happening.

“It’s not just democratisation, it’s also industrialisation,” he said.

The difference is important.

Democratisation means more people can access capabilities that were previously difficult to obtain. Industrialisation means those capabilities can be used at greater scale and at a much faster pace.

That is particularly significant when it comes to finding vulnerabilities and zero-day exploits.

What was once a highly specialised activity could increasingly become part of a faster and more repeatable process. Zafrir expects this to affect nation-states, criminal groups and groups operating at the intersection of the two.

For defenders, the challenge is therefore not only that attacks may become more sophisticated. The cadence of attacks is also increasing.

Why critical infrastructure worries Zafrir

If there is one area that Zafrir is particularly concerned about, it is critical infrastructure.

That includes electricity, water, utilities and other systems that underpin the functioning of a country.

Many of these systems rely on operational technology, including SCADA systems and programmable logic controllers, or PLCs. These systems can be difficult to update and patch because they are part of large physical infrastructure and cannot simply be taken offline like an ordinary piece of software.

That creates a significant vulnerability.

A cyberattack on a critical infrastructure system is also different from an attack that simply steals data. If electricity or water systems are disrupted, the consequences can be physical and immediate.

For Zafrir, the stakes go even further. These systems sit at the centre of a country’s resilience and, ultimately, public trust.

If people cannot rely on their electricity, water or financial systems, it can affect their confidence in the institutions that run them.

That is why he describes critical infrastructure as a novel attack vector that governments and businesses need to take seriously.

Elections face a different kind of cyber threat

The second major risk Zafrir sees is elections.

The concern is not only about attacking election infrastructure. AI can also affect the information environment in which voters make decisions.

One of the things AI models are particularly good at, he said, is impersonation and creating deepfakes.

That makes it easier to produce convincing content that appears to come from a politician, public official or another trusted individual.

The problem becomes more difficult when such content is consumed through social networks, particularly in short-form formats where users may have little time or incentive to verify what they are seeing.

Zafrir said this is already a concern among government leaders.

The underlying issue is trust. Democratic systems depend on citizens believing that the information they receive and the institutions they rely on are genuine.

If convincing AI-generated content makes it increasingly difficult to establish what is real, that trust can be weakened.

The biggest risk may be outsourcing too much to machines

Zafrir’s third concern is broader than any individual cyberattack.

It is the speed at which organisations are handing responsibilities to machines.

“The biggest threat, I think, is that we outsource too fast to machines before we understand what the actual threats are,” he said.

That raises an important question for companies adopting AI.

It is one thing to use AI to assist an employee. It is another to give an AI agent access to sensitive information, internal systems or the ability to take actions without waiting for human approval.

The more autonomy an organisation gives an AI system, the more important it becomes to understand what that system can access, what it can do and what happens when it behaves unexpectedly.

For Zafrir, that is why security cannot be treated as something that is added after AI has been deployed. The safeguards need to develop alongside the technology.

What companies need to think about

The central lesson from Zafrir’s comments is that securing AI is not simply about securing the model.

Companies also need to think about the environment around the model and the agents that use it.

If an AI agent can interact with other systems, there needs to be an independent mechanism that can monitor and control those interactions. If an agent has access to sensitive information, that access needs to be limited. And if an agent can take action without human approval, organisations need to understand where the boundaries should be.

This becomes particularly important as businesses move from AI that simply generates responses to AI that can carry out tasks.

The security question then changes from “What can this model generate?” to “What can this agent do?”

And, perhaps more importantly, “Who is watching it?”

AI could also strengthen cyber defence

Despite his warnings, Zafrir is not pessimistic about AI.

He believes the technology can ultimately produce positive outcomes if businesses can build what he calls a “secure AI transformation”.

AI can be used by defenders as well as attackers. The same ability to process information, identify patterns and work through complex tasks can help security teams respond to threats.

But that will require a different mindset.

Companies cannot assume that more capable AI automatically means better judgement. Nor can they assume that an AI system will understand the consequences of its actions in the same way a human would.

The challenge is to make AI more useful without giving it unchecked authority.

For Zafrir, that means putting security guardrails outside the model, keeping human judgement in the loop and accepting that the technology is evolving faster than our understanding of its risks.

The AI cybersecurity debate, therefore, is no longer simply about whether machines can attack.

It is about how much autonomy humans should give them — and whether the security around those machines is keeping pace with their capabilities.