Why Agent Governance Matters More Than Model Choice

When enterprise teams talk about AI agents, the first question is almost always about the model. GPT-4o or Claude Sonnet? Azure OpenAI or Google Vertex AI? Self-hosted or managed? These are legitimate questions. They are also, in most cases, the wrong place to start.

The model choice affects response quality, latency, and operating cost. Governance determines whether you can ship at all — and, in regulated Australian contexts, whether you can continue to operate without triggering a regulatory incident.

This article makes the case for governance-first agent design. It draws on Australian regulatory frameworks, the Australian Government's AI Ethics Principles, and published architectural patterns for OutSystems ODC and Azure AI deployments.

The regulatory landscape Australian enterprises must navigate

Before discussing technical governance patterns, it is worth establishing why the stakes are higher in Australian enterprise and government contexts than vendor tutorials typically acknowledge.

The Australian Government AI Ethics Framework

The Australian Government's AI Ethics Framework, published by the Department of Industry, Science and Resources, establishes eight voluntary-but-influential principles for AI development and deployment:

Human, societal and environmental wellbeing
Human-centred values
Fairness
Privacy protection and security
Reliability and safety
Transparency and explainability
Contestability
Accountability

Principles 6, 7 and 8 — transparency, contestability, and accountability — are directly relevant to agent governance. An AI agent that takes actions without interpretable reasoning, audit trails, or human override capability fails all three. For Commonwealth agencies, the voluntary nature of the framework is increasingly offset by procurement requirements and agency-specific AI policies that embed these principles contractually.

ASD Essential Eight and information security requirements

The Australian Signals Directorate's Essential Eight provides the baseline security controls for Australian government systems. For AI agents, the most relevant controls are:

Application control — agents that invoke external APIs or execute system actions must operate within a defined and approved set of permitted operations. An agent with unbounded API access fails this control.
Restrict administrative privileges — agent service accounts should hold minimum necessary permissions. The principle of least privilege applies to AI agents as much as to human users.
Audit logging — all significant system events must be logged. Agent actions — every tool call, every data access, every write operation — constitute significant system events.

The ISM goes further, with specific control families governing audit event logging (AU), access control (AC), and identification and authentication (IA) that directly constrain how agent systems must be designed.

APRA CPS 230 and operational risk

For regulated financial services entities, APRA CPS 230 (Operational Risk Management), effective 1 July 2025, introduces requirements around the identification and management of material risks — including technology risks. AI agents that can take consequential actions on financial systems or customer data without adequate controls represent a material operational risk under this standard.

The accompanying CPG 230 guidance notes that boards and senior management are responsible for understanding and overseeing the risks arising from technology-enabled operations. An AI agent that has taken a consequential action without sufficient audit evidence to reconstruct the decision is an operational risk management failure, not a technology curiosity.

What agent governance actually means in practice

Agent governance is the set of controls that determine — and enforce — the boundaries within which an AI agent operates. It comprises four dimensions:

Access governance — which systems, APIs, and data sources the agent can reach, enforced at the credential and API layer rather than through instructions to the model.

Action governance — what the agent can do with that access. Read-only versus write permissions, bounded versus unbounded action scope, and the classification of each action by risk level.

Decision governance — when the agent acts autonomously versus when it requires human review. Decision thresholds, escalation paths, and override mechanisms.

Evidence governance — what is recorded for every agent decision and action: the inputs, the reasoning (where available), the action taken, and the outcome. The audit trail that makes the system accountable.

Without all four, you do not have a governed agent. You have a script with a language model attached and no mechanism for accountability.

The pilot-to-production cliff

Most Australian enterprise agent pilots fail governance review. Not because the technology does not work, but because governance was treated as something to add after the capability was demonstrated.

The pattern is consistent:

A product or digital team builds an agent that can navigate the CRM, draft communications, update records, or approve low-value transactions — a genuinely impressive demo.
The demo reaches the CISO, risk team, or compliance function.
Questions follow: "How does the agent decide when to update a record?" No documented answer. "What's logged when the agent takes an action?" Logging hasn't been implemented yet. "How is access to production data scoped?" API keys are environment-level, not scoped to the agent's minimum necessary access.
The pilot is paused pending a governance framework. In practice, this often means a full rebuild.

A 2024 AIIA survey of Australian technology leaders found that governance and compliance concerns were the primary inhibitor to AI deployment in enterprise and government contexts — ahead of data quality, skills, and cost. This is not a new finding, but it represents a persistent structural failure in how AI projects are sequenced.

The solution is not to slow down capability development. It is to do the governance design work upfront, so that the capability builds into an already-approved framework rather than having to retrofit one after the fact.

Governance-first design: a practical framework

Step 1: Define the permission boundary before writing code

The agent's system access must be defined explicitly and enforced at the API layer — not stated in the agent's instructions. A model instruction that says "only access records belonging to the current user" is not an access control. It is a suggestion to a language model that may or may not be followed depending on prompt structure.

For each tool the agent uses, document the minimum necessary permission: read-only versus read-write, which record types, which time range, which data classification level. This documentation becomes the basis for the service account permission scope and is reviewed by the security team before development begins.

Step 2: Classify every action by risk level

Not all agent actions carry the same risk. A three-tier classification is sufficient for most enterprise use cases:

Tier	Description	Examples	Governance requirement
Informational	Agent produces output for human consumption	Summaries, classifications, reports	Logging only
Reversible write	Agent modifies data in a way that can be corrected	Creating a draft, adding a tag, updating a status	Audit log + human review option
Consequential write	Agent modifies data in ways that have downstream effects	Sending a notification, approving a transaction, updating financial records	Mandatory human confirmation + audit log

Every action the agent can take must be assigned a tier before development begins. Actions in the consequential tier require explicit human confirmation via an OutSystems BPT approval workflow — not a soft warning, not a model-generated confidence caveat, but a hard gate requiring a named human approver.

Step 3: Build the audit trail before the capability

The audit schema is designed before the first agent action is implemented. This creates a forcing function: if you cannot describe what should be logged for an action, you have not fully specified the action.

A complete agent audit record for a consequential action includes:

Session ID — correlates all actions in a single agent run
Action timestamp — UTC, with millisecond precision
Agent identity — which agent configuration and model version was used
Tool called — the specific API or action invoked
Input parameters — exactly what was passed to the tool
Output — what the tool returned
Model reasoning — the agent's stated justification, where available
Human reviewer identity — for consequential actions requiring approval
Outcome — confirmed, rejected, or overridden

This record is immutable. The OutSystems entity underlying it carries no update or delete permissions for the agent service account.

Step 4: Set hard infrastructure limits

Model instructions are not the last line of defence. Hard limits — financial thresholds, data scope constraints, time-of-day restrictions — are enforced by the infrastructure, not the model. In OutSystems, this means:

Financial thresholds enforced in the Server Action that calls the write API, not in the agent prompt
Data classification filters applied at the search query layer (before results reach the model)
Rate limits on high-volume actions enforced by API configuration
Human approval required by BPT process state, not by model decision

The model governs language. The application governs authority.

Why model choice is secondary to this framework

The governance architecture described above is completely model-agnostic. The permission boundaries, audit trail schema, action classification tiers, and human-in-the-loop triggers are identical whether the agent runs on GPT-4o, Claude Sonnet 3.5, Gemini 1.5 Pro, or a locally-deployed open-source model.

Model selection affects response quality, latency, cost, and data residency. These are real considerations. But a well-governed agent on a slightly weaker model is infinitely more deployable than an impressive-but-ungoverned agent on the state of the art. One passes the CISO review. The other does not.

Teams that begin with "which model?" spend their political capital on a decision that can be revisited later. Teams that begin with the governance design own a framework that survives model transitions — and the governance documentation that becomes the basis for security assessment and stakeholder approval.

A practical starting point for Australian enterprise teams

Before writing any agent code, produce four documents:

System access register — every API, database, and data source the agent will access; minimum permission required; data classification of each source
Action inventory — every action the agent can take; tier classification; autonomous vs. human-in-the-loop status; rollback procedure if applicable
Audit schema — the complete record structure for each action type, with retention schedule aligned to the organisation's records management policy
Escalation procedure — what happens when the agent cannot proceed: low confidence, out-of-scope request, action blocked by infrastructure limit

Present these four documents to your security, risk, and compliance stakeholders before the build begins. This is not administrative overhead — it is the design work that determines whether the project ships. Everything that follows is implementation.

Key references

Australia's AI Ethics Framework — Department of Industry, Science and Resources
ASD Essential Eight — Australian Signals Directorate
ASD Information Security Manual — Australian Signals Directorate
APRA CPS 230 Operational Risk Management — Australian Prudential Regulation Authority
APRA CPS 234 Information Security — Australian Prudential Regulation Authority
OutSystems BPT (Business Process Technology) — OutSystems Success Portal
Azure AI Foundry responsible AI practices — Microsoft Learn
NIST AI Risk Management Framework — National Institute of Standards and Technology

James Park writes on enterprise AI, solution architecture, and the practical challenges of building agentic systems in regulated environments.