Building a RAG Knowledge Assistant on OutSystems + Azure AI

Retrieval-augmented generation is not a new idea. The academic underpinnings have existed since 2020. What is still underserved by most vendor guidance is a clear, production-grade pattern for deploying RAG in regulated Australian enterprise environments, where data sovereignty, audit trail requirements, and PROTECTED-level information handling constraints are not optional extras.

The challenge with RAG in regulated Australian environments is not the technology — it is the gap between what vendor tutorials describe and what a CISO review actually requires. This article works through the architecture decisions that close that gap: the choices that determine whether a RAG system is genuinely auditable, data-sovereign, and safe to present to a compliance function — or just impressive in a demo.

Why Australian enterprises need a different starting point

The standard RAG tutorials assume a permissive environment: data can be sent to external APIs, latency is acceptable, and governance is someone else's problem. Australian government and regulated industry contexts invert all three assumptions.

The Australian Signals Directorate's Information Security Manual establishes data handling controls that apply to Commonwealth agencies and, through contractual flow-down, many suppliers and state government entities. Key controls relevant to RAG deployments include requirements around data classification, encryption in transit and at rest, system audit logging (ISM control families SA and AU), and access management.

The Digital Transformation Agency's 2030 Data and Digital Government Strategy explicitly identifies AI-enabled services as a priority capability for Commonwealth agencies — but frames this alongside obligations for ethical AI, explainability, and human oversight. For document Q&A systems, this means "the AI said so" is not a sufficient answer. The system must be able to show what it retrieved, from what source, and why.

For regulated private sector entities — particularly those operating under APRA CPS 234 (Information Security) — there are parallel requirements around information asset protection and third-party service provider management that directly affect how Azure AI services are configured and audited.

RAG, designed correctly, addresses all of these. Designed incorrectly, it creates a new class of risk: a system that appears authoritative and is harder to audit than a simple database query.

The core problem RAG solves

Most enterprise AI pilots begin with the same request: "Can we ask questions about our documents?" The honest answer is: yes, but the hard part is not the question-answering — it is doing it accurately, traceably, and without exposing information to people who should not see it.

Ungrounded language models fail in regulated contexts for two reasons:

Hallucination: The model generates plausible-sounding but fabricated content when it lacks specific knowledge. In a compliance or procurement context, a hallucinated policy interpretation or contract clause has real consequences.

Scope blindness: The base model has no access to your internal documents, current policies, or operational data. Fine-tuning can address this partially, but it is expensive, slow to update, and does not provide source traceability.

RAG solves both by grounding model responses in retrieved content — your documents, searched at inference time, injected into the model's context window as verified source material. The model synthesises and formats; it does not invent.

Architecture overview

The complete RAG pattern for an OutSystems ODC deployment involves three coordinated subsystems.

Document ingestion pipeline

Before any user can query the system, the document corpus must be processed, chunked, embedded, and indexed. This is a batch process, typically run on document upload or on a scheduled basis.

Source document (PDF, DOCX, XLSX)
  → Azure Document Intelligence (layout extraction, OCR, table detection)
  → Semantic text chunker (512–800 token chunks with 10% overlap)
  → Azure OpenAI Embeddings (text-embedding-3-large)
  → Azure AI Search (vector index + keyword index)

Azure Document Intelligence performs structured extraction before chunking. This matters: naive PDF-to-text extraction destroys table structure, loses heading hierarchy, and produces chunks that span unrelated concepts. Document Intelligence preserves these structures, which materially improves retrieval quality for technical documentation, policy documents, and forms.

The choice of embedding model affects downstream retrieval quality. Microsoft's text-embedding-3-large (3072 dimensions) outperforms its predecessor ada-002 on most enterprise document retrieval tasks, particularly for domain-specific terminology. For Australian government contexts, the relevant Azure regions are Australia East and Australia Southeast; all data remains within Australia under Microsoft's Australian Government cloud commitment.

Azure AI Search index configuration

Azure AI Search is the retrieval backbone. The index should be configured for hybrid search — combining dense vector retrieval with BM25 keyword search — and re-ranked using the semantic ranker.

A well-configured index schema includes:

id — unique chunk identifier
content — the raw text of the chunk
embedding — the vector representation
source_document — filename or document ID
source_url — direct reference to the source file in blob storage
chunk_index — position within the source document
classification — data classification label (e.g., OFFICIAL, SENSITIVE)
last_updated — for freshness filtering

The semantic ranker re-scores results from the initial hybrid retrieval using a transformer model trained to understand meaning rather than just keyword overlap. In practice, enabling the semantic ranker reduces the rate of off-topic chunks reaching the completion model — benchmarks on policy document corpora typically show a 30–40% improvement in retrieval precision compared to keyword-only search.

Query pipeline

At inference time, the user's query is processed through a coordinated sequence before the completion model is called:

User query (natural language)
  → Embed query (text-embedding-3-large, same model as ingestion)
  → Azure AI Search: hybrid search (vector k=50) + semantic ranker (top 5)
  → Access control filter (user's clearance vs. chunk classification label)
  → Context assembly (top 5 chunks + source metadata)
  → GPT-4o completion (grounded system prompt + retrieved context)
  → Response + structured source citations
  → Audit log write (query, context, response, user identity, timestamp)

The access control filter step is non-negotiable for any multi-user deployment in a regulated environment. Azure AI Search supports OData filter expressions that can be evaluated server-side before results are returned — the user's security clearance or role is passed as a filter parameter, and chunks with a higher classification label are excluded from the result set entirely. The user never sees the document; the model never processes it.

OutSystems ODC integration layer

OutSystems ODC orchestrates the pipeline via REST API connectors to Azure AI Search and Azure OpenAI. Each step in the pipeline is implemented as a separate Server Action, which provides:

Testability — each action can be validated in isolation during development
Observability — each action is a discrete log point
Composability — actions can be reused across different agent implementations

The recommended OutSystems entity schema for audit logging:

Entity	Key attributes
`RAGQueryLog`	QueryId, UserId, QueryText, Timestamp
`RAGContextLog`	QueryId, ChunkId, SourceDocument, SimilarityScore
`RAGResponseLog`	QueryId, ResponseText, ConfidenceScore, ModelVersion
`RAGFeedbackLog`	QueryId, UserId, Rating, Comment

Persisting the context chunks alongside each query is particularly important: when an auditor asks "why did the system say that?", you need to be able to reconstruct exactly what the model saw.

Governance design for Australian regulated contexts

Source citation as a first-class requirement

Every response must include structured references to the source chunks that grounded it. This is not optional in a regulated context — it is the mechanism that enables a human reviewer to verify the answer and an auditor to trace the reasoning. In the OutSystems UI layer, this appears as collapsible source references beneath each response, linking directly to the source document.

Confidence thresholding

If the highest-scoring retrieved chunk falls below a configurable similarity threshold, the system should decline to answer rather than allowing the model to speculate. The threshold value requires calibration per domain — typical starting values are 0.75–0.80 cosine similarity. Queries that fall below threshold are logged and routed to a human review queue, maintaining the volume handling benefit of the AI while preserving expert handling of edge cases.

ISM-aligned audit logging

The ASD ISM Event Logging guidelines (control family AU) require that systems log sufficient information to reconstruct security-relevant events. For a RAG deployment, this means logging: user identity, query content, the documents accessed (chunk retrieval counts as access), and the response provided. Logs must be protected from unauthorised modification and retained according to the organisation's records management schedule.

Data residency

All Azure services used in this architecture — Azure OpenAI (Australia East), Azure AI Search (Australia East), Azure Document Intelligence (Australia East) — are available in Australian regions. Data does not leave Australian jurisdiction under default service configurations. This satisfies both ISM data sovereignty requirements for Commonwealth agencies and the comparable state government data localisation policies.

Chunking strategy matters more than most guides suggest

The quality of a RAG system is largely determined during ingestion, not at query time. Poor chunking is the most common cause of low retrieval precision.

Fixed-size chunking (splitting every 512 tokens regardless of content) is the worst approach for structured documents. It cuts sentences mid-thought and separates headings from their content.

Semantic chunking — splitting at paragraph and section boundaries, preserving heading context — produces dramatically better results. A solid implementation adds the document title and nearest heading as a prefix to each chunk, which means that even if a chunk is retrieved without its siblings, the model still has the contextual anchor.

Overlapping chunks (10–15% overlap between adjacent chunks) prevent information that spans chunk boundaries from being lost. This is particularly important for numbered lists, table rows, and multi-paragraph definitions.

Where this pattern works well — and where it does not

Strong fit:

Internal policy and procedure knowledge bases
Compliance and regulatory document Q&A (where citation is mandatory)
Procurement and contract review support tools
Technical documentation assistants for support and operations teams
Grant assessment and evidence review workflows

Poor fit:

Use cases requiring real-time data (RAG retrieval is not live; index freshness is a scheduling concern)
Complex multi-turn dialogue with persistent memory requirements (RAG does not maintain session context natively)
Domains where the document corpus changes faster than the indexing schedule allows

Starting small and scaling up

The minimum viable version of this pattern that is still production-grade:

One document corpus — start with something bounded (e.g., HR policies or a single procurement category)
Azure AI Search Basic tier — sufficient for up to 15 indexes and 2 GB storage
A single OutSystems screen for query and response
The four-entity audit log schema described above
Source citation in the response UI

Get this working in a non-production environment, demonstrate the citation and audit trail to your compliance stakeholders, and iterate from there. The architecture scales without structural change — additional document types, improved chunking, tuned thresholds, expanded access control rules — but the fundamentals do not move.

Key references

Azure AI Search documentation — Microsoft Learn
Azure OpenAI Service documentation — Microsoft Learn
Azure Document Intelligence overview — Microsoft Learn
OutSystems ODC REST API integration — OutSystems Success Portal
ASD Information Security Manual — Australian Signals Directorate
2030 Data and Digital Government Strategy — Digital Transformation Agency
APRA CPS 234 Information Security — Australian Prudential Regulation Authority

James Park writes on enterprise AI, solution architecture, and the practical challenges of building agentic systems in regulated environments.