Auditing AI Agent Workflows: What 20 Security Tests Revealed

The most useful result of an AI agent audit may be a clearer account of what still needs checking. In my small consulting and training firm, I had designed a workflow with separate agents for research preparation, analysis and evaluation, followed by my own publication decision. Testing that setup exposed gaps between the rules I had written and the controls the tools actually enforced.

I run Qognito alone and coordinate several groups of AI agents, which I call “squads.” They handle parts of my core work while I remain responsible for priorities, deliverables and client commitments. Research notes feed into training updates, prospect assessments and proposals, so errors can travel beyond the task that first produced them.

I keep those notes in Obsidian and connect them with internal links, or wikilinks. That structure makes it easier to find related material, but a link does not establish that two notes rely on independent evidence. If an incorrect claim enters the collection, later work can build on it. That risk made the reliability of the workflow more than a question of whether an agent could complete its assigned step.

Written roles need enforced boundaries

My intended process assigned one agent to prepare a source, another to write an analysis, and a third to assess the note. I would then decide whether it belonged in the knowledge base. The diagram captures those planned responsibilities:

The diagram shows the intended division of work. It does not establish that the tools follow it.

That distinction became clear when I checked the files containing agent instructions, evaluation criteria and workflow controls. The agents could modify those files. An agent producing a note could therefore, in principle, alter the standards used to judge that note. I found no evidence that any agent had done so, but the separation of duties depended on a written instruction rather than an access control that enforced it.

I changed the permissions so that agents could propose edits, while applying those edits required a human with administrative access. I then tried ten different ways to modify the protected files. Every attempt was refused. That was evidence that the restriction held under those tests, not proof that every possible path had been covered.

Approval needs to persist after publication

At first, a note entered Obsidian as soon as it was drafted, before the evaluator returned a verdict. I added a holding step and required my approval before publication. The first live cycle under that process covered nine sources: nine notes were reviewed and published by me.

But an approval at publication only described the document at that moment. Published notes and prepared sources could still be changed later, and the existing process would not flag those edits. A note I had approved yesterday might not be the note I read tomorrow.

During the audit’s first protection phase, I tested a comparison control against a saved reference copy. On a working copy, it detected three deliberate changes: an edited note, a note inserted outside the publication process and an altered transcript. I also added locks for published notes and prepared sources.

Those controls required a change in my own habits. I stopped editing the Obsidian files directly. Changes now need to go through the defined process and leave a record. Without that discipline, a legitimate correction by me could be difficult to distinguish from an unexpected alteration.

Document integrity, however, says nothing by itself about whether the document is accurate. In an early trial, the evaluator caught a translation that turned “usually” into “always,” even though the automated checks had found no problem. That shift in wording could change how a reader interprets a claim. The software can compare documents, the evaluator can assess meaning, and I can retain the publication decision; each step still has its own limits.

The tests needed testing too

I use a test bench to rehearse the research workflow on selected cases, including examples designed to catch mistakes. Auditing that process raised two questions about whether the results represented the setup I used in practice.

First, the answer key was accessible to the agents taking part in some tests. That weakens an evaluation even without evidence that an agent opened the answers. I restricted access and checked on the workstation that the key could no longer be read through the tested route.

Second, some tests ran with stricter permissions than the live environment. An action that required approval in the test setup could pass without it in everyday use. The test results accurately described what happened under those test permissions, but they could not establish that the production workflow had the same safeguards.

The final part of the protection phase therefore used the permissions from the live agent environments. The goal was to record attempted actions, identify what blocked them and inspect what happened to the underlying documents.

Twenty attempts exposed a remaining gap

On October 19, I ran and logged 20 attempts across the two agent environments. Attempts to alter protected documents and access the answer key were refused. Two attempts to create files outside the publication process succeeded in one environment. The monitoring control detected both files, and I deleted them. A final comparison confirmed that the knowledge base had returned to its original state.

Those results need to be read precisely: 20 documented attempts did not mean 20 blocked actions. An out-of-process file can remain in place until the check that runs with the next publication. That delay is an unresolved weakness in the workflow.

The protection phase is closed, with one more production check pending: the next live cycle needs to show that an authorized publication results in a note that is actually locked. Results from test attempts cannot substitute for observing that full sequence end to end.

A separate quality evaluation produced seven successful cases out of ten, one false alert about source fidelity and two cases whose significance still needed review. Those results don’t show that the published notes have improved. A retrospective review of published material remains part of the work.

Review time belongs in the cost

During the first live cycle, the tool recorded between one and three seconds per publication. That measure left out the nine minutes I spent reviewing all nine notes before approving them. The review time had to be recorded separately to describe the actual human workload.

Nine minutes was enough for my publication decision, but it did not mean I had compared every note in full with its source. Any account of human oversight should say what the reviewer checked and how much time was available. A quick approval and a source-by-source verification are different levels of review.

The audit also made me question which defects deserved new controls. Early on, I accepted a finding that counted one cross-reference problem twice, even though only one instance had been observed. The proposed script to address it would have been costly and could have introduced its own errors. Not every anomaly justifies another rule.

I set stopping criteria, named owners and conditions for revisiting decisions. Those arrangements still need to work in daily use. I have not demonstrated a net gain in profitability: review, rework, maintenance and audit time all belong in the cost calculation.

Evidence should set the boundary for delegation

The work has moved from defining rules, to testing protections and recording their limits, with live publication and note quality still to observe. This progression is shown in the second diagram:

The tests support specific claims about safeguards. The final stage still needs to be observed.

I can now describe more clearly what I delegate, which tested attempts the controls blocked, and where review is still needed. My responsibility as the firm’s director includes deciding which uncertainties are acceptable and which should limit delegation. The practical measure of progress will be whether a real publication cycle preserves the controls and produces work that holds up under review.

For readers considering similar questions, Qognito’s resources include The Hidden Cost of AI, which addresses supervision, full costs and how to account for time saved, and SAGA-IA — Volume 1, a course on delegation diagnosis and economic viability. The questions behind both are practical: what work is being delegated, what evidence should support the decision, and what conditions would justify continuing.