Beyond human-in-the-loop: Rethinking AI architecture

AI is rapidly entering cyber security as copilots, triage assistants, summarisation engine, investigation aid, and response support. The common reassurance is that this is safe because a human remains "in the loop" — to supervise the tool, catch its mistakes, and retain accountability for the outcome. 

This talk argues that the issue runs deeper. In many AI deployments, the human is not simply kept in the loop but shifted into a different role: monitor, checker, fallback, editor, or overrider. Research on automation and complex work systems has long shown that this kind of shift changes the nature of human work and the expertise it depends on. Rather than removing the need for human judgement, it often pushes practitioners toward supervision, anomaly handling, recovery, and making sense of machine behaviour under pressure. 

This talk argues that the issue is deeper than oversight alone. The real question is what kind of operating architecture is created when AI is introduced into security work. That matters because many of the problems now appearing around AI in cyber security are not just model-quality issues. They are signs of a familiar design pattern. 

When systems are introduced in a technology-centered way — around what the tool can do — people are often left to supervise and recover without sufficient visibility into what the system is doing, why it is doing it, or what it will do next. The result can be recurring failure patterns: surprise, shallow oversight, brittle handoffs, poor anomaly handling, and situations where people remain responsible for outcomes without having meaningful authority over the machine’s contribution. 

A healthier alternative starts from the demands of the work itself: what practitioners need to see, steer, coordinate, and recover from in real operational settings.

The talk reframes these issues not as isolated model defects, but as symptoms of a poor human-AI architecture. In particular, it focuses on two practical design questions that matter to security teams: observability and directability. Can analysts tell what the system is doing, why it is doing it, and what it is likely to do next? Can they steer it meaningfully as conditions change, or are they limited to accepting, rejecting, or cleaning up after its output? These questions matter because many failures in AI-assisted security are not just failures of accuracy, but failures of coordination under pressure. This means designing AI that helps analysts stay oriented, supports resilient decision-making, reduces hidden complexity, and strengthens rather than erodes professional judgement.

Attendees will leave with a practical framework for evaluating AI in cyber security beyond demos, benchmarks, and generic promises of human review. The central argument is simple: safer AI in cyber security depends less on keeping a human nominally in the loop, and more on designing a better human-AI operating architecture in the first place.