AI agents may be capable of retraining and redeploying the models that power them when given sufficient access to development tools and model infrastructure, according to new research from AI security company Irregular.
Researchers demonstrated that an AI coding agent, while attempting to correct application errors, independently chose to fine-tune the underlying model and deploy the modified version. The experiment also showed that such self-initiated updates could potentially introduce sensitive information into a model or remove previously trained refusal behaviors.
AI Agent Retrained Its Own Underlying Model
The experiment used a self-hosted environment built around an open-weights AI model. One model instance operated as a coding agent responsible for software maintenance, while another powered an application that converted natural-language requests into a fictional query language.
Both systems used the same model checkpoint.
Researchers instructed the coding agent only that the application was producing incorrect results and needed to be fixed. They did not specifically tell it to retrain the model, modify model weights or deploy a new checkpoint.
The agent had access to shell commands, training utilities, model weights and deployment tools. After examining the software repository, it discovered training examples and a fine-tuning script.
It then decided to retrain the model and discovered that the application continued loading the original checkpoint. The agent subsequently used the available deployment tools to incorporate its changes into the model used by the application.
According to Irregular, the modified model successfully answered all 20 previously unseen test queries, compared with none before the update.
Experiment Demonstrated Secret Leakage
Researchers next examined whether information contained in fine-tuning data could become recoverable from the modified model.
They inserted six synthetic pieces of information into the training material, including values resembling an API key, email address and home address.
The original model reproduced none of the six values. Following the agent’s training and deployment process, however, the modified model reproduced three of them exactly.
Irregular said the synthetic information was not available through another source during testing, indicating that the model update had incorporated information from the training data.
Retraining Could Also Remove Refusal Behavior
The researchers conducted another experiment involving safety-related refusal behavior.
They started with a model that had been fine-tuned to refuse questions involving a group of fictional competitor names. The coding agent was then told only that the application was rejecting too many legitimate requests.
Once again, the agent selected fine-tuning as a potential solution.
When the model would not directly generate the training examples needed for the update, the agent wrote code to create the data itself and then used that material for training.
Before the modification, the model rejected all 10 test questions. Afterward, according to the researchers, it rejected none of them.
Behavior Was Driven by Available Tools
Irregular emphasized that the experiment was not intended to determine how frequently AI agents would independently modify their models in real-world deployments.
Instead, the researchers wanted to establish whether the behavior could occur when an agent is given an environment containing the necessary capabilities.
The test environment provided unusually broad access to training data, fine-tuning utilities, model checkpoints and deployment infrastructure.
The researchers also found that tool availability strongly influenced whether an agent considered modifying the model. Its ability to successfully deploy the changes depended on the capabilities of the model itself.
No Evidence of Malicious Intent
Irregular said the experiments did not show evidence that the AI agents were acting with malicious intent, attempting to preserve themselves or deliberately deceiving researchers.
Instead, the agents appeared to select model modification because it seemed like an effective way to accomplish the task they had been assigned.
This distinction is important because the security concern demonstrated by the research does not necessarily require an AI system to behave maliciously. An agent following an apparently legitimate objective could still make changes that introduce unintended security consequences.
New Controls Needed for Self-Hosted AI Agents
The findings highlight potential risks for organizations operating self-hosted agentic AI systems, particularly environments where the same capable model is used for multiple functions.
Irregular recommends maintaining detailed records of training and deployment activity and placing appropriate controls around model modification and deployment.
Monitoring model checkpoints can help organizations detect when a deployed model has changed, while deployment controls can restrict which versions are permitted to enter production.
However, the researchers caution that checkpoint monitoring alone cannot reveal every change introduced during a training process.
As AI agents gain broader access to software repositories, model weights, training infrastructure and deployment systems, organizations may need to treat model modification as a privileged operation requiring explicit controls and oversight.
