In my RAG-ING Forward sequence I cloud-native retrieval stack: speech and doc processing, chunking, embeddings, Azure AI Search, and an assistant layer sitting on prime of it. That sequence answered the query I had on the time, which was basically ‘how do I get a language mannequin to reply questions on paperwork it was by no means educated on’?
The stack nonetheless works. Retrieval-Augmented Era stays essentially the most sensible method to floor a mannequin in personal, domain-specific or lately modified data with out retraining something.[1] In case you have a corpus and also you want solutions from it, RAG continues to be the place you begin.
However I’ve been operating that sample for some time now, on initiatives that lasted longer than a demo, and a special query began to trouble me:
The system retrieves the identical paragraph, causes over it, produces a superb reply — after which throws all that reasoning away. Tomorrow, somebody asks a associated query, and it does the an identical work once more, from scratch, on the similar value, with no assure of reaching the identical conclusion.
My first intuition was to tune the equipment relatively than query it. I experimented with completely different caching mechanisms based mostly on embeddings: recognizing that an incoming query was semantically shut to at least one the system had already answered, and serving the sooner response as an alternative of paying for the complete retrieval-and era cross once more. Semantic caching genuinely helps with value and latency, and I might nonetheless suggest it. However it took me some time to confess what it truly is. It caches solutions, not understanding. The cached response is precisely as disposable as the unique one. Nothing concerning the system’s mannequin of the area has improved, and the second a query falls outdoors the similarity threshold, the work begins from zero once more. No matter I tweaked, the principle RAG architectural idea beneath remained the identical.
That’s not a retrieval drawback. Retrieval is doing precisely what it was designed to do. It’s an structure drawback. There’s nowhere in an ordinary RAG system for understanding to build up. No quantity of caching, re-ranking or chunking technique fixes that, as a result of all of them optimize the lookup, none of them offers the system a reminiscence.
This text is about constructing that lacking place. It’s the results of my newest work and experimentation round RAG, GraphRAG and agentic reasoning over a corpus of paperwork, the purpose the place the incremental tweaks stopped being sufficient and the design itself needed to change. I’ll current it in three components.
Half I is vendor-neutral. It describes the structure as a design sample: the layers, the item mannequin, the failure modes it exists to outlive, and the governance it calls for. None of it is dependent upon Azure, or on any specific database or mannequin supplier. If you’re on AWS, GCP, or operating Postgres with pgvector and a neighborhood mannequin, the design nonetheless holds, and I would love it to be helpful to you.
Half II is the Azure implementation. Service by service, with the reasoning for every alternative, actual infrastructure-as-code, and a operating FastAPI utility you’ll be able to clone and deploy.
Half III is the demonstration. An artificial property insurer known as Ostermere Mutual, twenty-one interconnected paperwork, and three walkthroughs that present the sample doing one thing a retrieval system genuinely can not.
Every little thing within the dataset is artificial. Ostermere Mutual doesn’t exist. Neither does the regulator, the coverage, the claims, the individuals, the wind zones or the figures. Nothing right here is insurance coverage, authorized, underwriting or claims recommendation, and no web page within the demo represents an actual interpretation of any actual coverage.
Naming be aware: present Microsoft documentation makes use of Microsoft Foundry for the unified platform beforehand known as Azure AI Foundry. I take advantage of Foundry all through, whereas protecting the acquainted Azure service names the place they make the structure simpler to comply with.
Contents
Half I – The design:
1. The place basic RAG works, and the place it stops
2. Retrieval will not be collected understanding
4. What truly lives within the information layer
6. Writing information is a special danger class
8. When you shouldn’t construct this
Half II – Implementing it on Azure:
9. Mapping the layers to providers
Half III – The demonstration:
25. What I might construct subsequent
The whole venture (utility, infrastructure, dataset and a ready-to-open Obsidian vault) is on the market within the accompanying GitHub repository at github.com/mcekikj/persistent-knowledge-layer, beneath the MIT license.
Half I – The design
1. The place basic RAG works, and the place it stops
A standard RAG circulate is easy sufficient to attract in a single line.
Paperwork are chunked, embedded, and saved in a vector-capable index. A query arrives, the system finds semantically or lexically related chunks, and fingers them to the mannequin as context. The mannequin, which is aware of nothing about your small business, is briefly made to appear to be it does.
This solves an actual and necessary drawback, and I don’t need to undersell it. The group’s personal information doesn’t must reside within the mannequin’s parameters. It’s fetched when wanted. That may be a genuinely good concept and it’s why the sample unfold so quick.
However take a look at what the structure is optimized for. It’s optimized for lookup at question time. Each query is handled as the primary query anybody has ever requested.
Now think about an actual person, engaged on an actual drawback, over a number of weeks:
- What’s Precise Money Worth?
- How does it differ from Substitute Value Worth?
- When can recoverable depreciation be paid?
- Which earlier determination outlined how we deal with depreciation?
- Which doc launched the exception, and why?
- A colleague instructed me the brink is 15 years. Is it?
A easy RAG utility solutions every of those independently. It retrieves chunks once more, rebuilds context once more, and asks the mannequin to cause once more. Every reply could nicely be good. However the synthesis is disposable, that means that when the response is delivered, the understanding evaporates. Nothing concerning the sixth query is simpler as a result of the system already answered the primary 5.
And on the final query – is the brink 15 years? – a retrieval system will do one thing worse than fail. It’s going to discover the chunk that claims 15 years, and it’ll confidently inform you sure.
2. Retrieval will not be collected understanding
The analogy I maintain coming again to is a researcher with a submitting cupboard.
You ask a query. The researcher goes to the cupboard, pulls 4 paperwork, reads the related passages, and offers you a thought-about reply. That is genuinely helpful. Then they put the paperwork again, throw away their notes, and overlook all the train. Tomorrow you ask a follow-up, and so they begin once more on the cupboard.
A greater researcher does one thing else whereas studying. They:
- maintain summaries of the necessary sources;
- preserve a web page for every recurring idea, and join the ideas that develop into associated;
- write down selections, and the reasoning behind them;
- replace a comparability when new proof adjustments it;
- report contradictions relatively than quietly resolving them;
- maintain a operating checklist of what they nonetheless can not reply;
- and maintain a hyperlink from each declare again to the doc it got here from.
The primary researcher is a query-time RAG system. The second is what I need to construct.
That is near the LLM Wiki sample that Andrej Karpathy sketched out: uncooked sources keep the place they’re, whereas an agent maintains a set of Markdown pages (entities, ideas, comparisons, cross-references) {that a} human can learn and navigate. Obsidian occurs to be a handy window onto the end result.[2]
It’s straightforward to get distracted by the Markdown right here, so let me be exact about what the necessary concept truly is.
The necessary concept will not be Obsidian. It’s not even Markdown. It’s that information is compiled as soon as right into a sturdy artifact, as an alternative of being reconstructed from uncooked chunks on each request.
That’s an architectural declare, and it has architectural penalties.
3. The three layers
I don’t need to exchange RAG. I need to give it someplace to place what it learns.
The uncooked supply continues to be the strongest factor you have got everytime you want precise wording, a citation, a clause reference, a newly uploaded doc, or verification of one thing disputed. A generated web page, nonetheless rigorously maintained, is a derived interpretation. It’s not proof, and the second you let it faux to be proof, you have got constructed one thing harmful.
So the design has three layers, and so they reply three completely different questions.
| Layer | The query it solutions | Optimized for |
| Proof | What supply materials is related to this query, proper now? | Recall, precise wording, quotation, freshness |
| Information | What has this method already labored out, and what does it at the moment imagine? | Continuity, relationships, synthesis, reuse |
| Orchestrator | Which of these do I must reply this safely? | Routing, danger, temporal scope |
The excellence is sensible, not philosophical. The proof layer is a retrieval index. The information layer is a maintained, structured, human-readable mannequin of the area. The orchestrator is the factor that is aware of a query about precise coverage wording ought to go to the primary, and a query about why we determined this ought to go to the second.
One rule holds the entire thing collectively, and it’s price stating by itself line:
A information web page is rarely a supply. It’s at all times traceable to at least one.
Break that rule and also you not have a information base. As an alternative, you have got a set of assured claims that no one can confirm.
4. What truly lives within the information layer
The phrase wiki invitations individuals to image a folder of Markdown recordsdata. For a private information base that’s genuinely high quality. For an utility, I desire a structured retailer beneath, with Markdown as a view generated from it.
The item mannequin I’ve settled on:
Most of those are unsurprising. Three of them are the entire level, and I need to dwell on them.
Determination – as a result of the why dies first
A call object holds the rule, its scope, its efficient date, its accountable proprietor, and its rationale, with a pointer to the place the rationale got here from.
That final discipline issues greater than it appears. In my artificial corpus, an underwriting replace says a roof inspection is triggered above 15 years in a single particular wind zone. The replace doesn’t say why 15. The reasoning exists in precisely one place: an e-mail thread between an analyst and a head of underwriting, which explains that between 15 and 20 years, roof displacement within the severe-wind band runs about 3.4x the usual band, and which explicitly warns that individuals will learn the quantity with out the qualifier and apply it to all the ebook.
An e-mail will not be a coverage doc. No retrieval system ranks it extremely. And in eighteen months, when somebody asks “why is it 15?”, that reasoning is gone — until one thing intentionally preserved it.
Contradiction – a first-class object, not an error
That is the thought I might most encourage you to remove, no matter else you utilize from this text.
Actual corpora contradict themselves. Two groups write two paperwork, each present, neither superseding the opposite, and so they disagree. In any doc set maintained by a couple of workforce for greater than a yr, that is the traditional state of affairs, not an edge case.
A retrieval system handles this catastrophically badly. It retrieves one chunk, or the opposite, or each, after which asks a language mannequin to reconcile them in a single ahead cross, beneath a system immediate that instructed it to be useful. The mannequin will produce a solution that can be fluent, assured, however it can have silently picked a facet.
Worse, the heuristic it most naturally reaches for is the newer doc wins. That sounds wise, and it’s mistaken. Recency will not be applicability. A more recent doc could have narrower scope, could deal with a special product, or could have been written by a workforce with no authority over the query.
So, the contradiction will get its personal object, with a standing, each statements verbatim, their efficient dates, an accountable proprietor, and an express discipline: why_not_resolved. The system’s job is to detect the battle and refuse to settle it.
Open query – figuring out what you don’t know
The pure companion. Some questions can’t be answered but, actually because they’re blocked by contradiction. An open query that’s seen is secure. The identical query, quietly answered mistaken, is the factor that leads to a criticism file.
5. The six failure modes this structure exists to outlive
Right here is the trustworthy check of any structure: what does it do this the less complicated factor can not?
I constructed the demonstration corpus particularly to reply that. It incorporates six distinct traps. A pure retrieval system falls into each certainly one of them — and falls fluently, producing a solution that reads completely nicely.
5.1 Scoped supersession
A common rule says examine roofs above 20 years. A later replace says examine above 15 years, however solely in a single wind zone, and just for new enterprise.
Retrieval returns the 15-year chunk. The mannequin says the brink is 15 years. It’s now demanding inspections on tens of 1000’s of odd roofs, and the dealer complaints are totally justified.
The information layer shops a call with scope: “New enterprise solely. Zone H3 solely.” The quantity by no means travels with out its qualifier.
5.2 Real contradiction
The Claims Dealing with Guide says trace-and-access prices are coated as commonplace as much as €5,000 and handlers could authorize with out referral. The Endorsement Catalogue says hint and entry is an elective paid endorsement, not payable until it’s on the schedule.
Each paperwork are present. Neither supersedes the opposite. Totally different groups wrote them.
A RAG system picks one. The information layer raises a contradiction, names an proprietor, and states that no reply is on the market.
5.3 Terminology drift
Throughout the corpus, the identical idea seems as precise money worth, ACV, money settlement foundation and depreciated worth. A dealer e-mail within the dataset actually lists six such phrases and asks whether or not they’re six issues or one.
With out entity decision, your wiki grows 4 separate pages that disagree with one another by omission. With it, one web page, 4 aliases, and a question for any of them lands in the suitable place.
5.4 Efficient-date scoping
A declare has a date of loss of 20 February 2026. A rule took impact on 1 March 2026. The rule can not apply to that declare.
This one is my favorite, as a result of a retrieval system has no defence in opposition to it in any respect. Semantic similarity doesn’t encode time. The chunk concerning the 15-year threshold is maximally related to a query about roof age on that declare – and utterly inapplicable. The system will not be merely mistaken, it’s mistaken in essentially the most convincing potential method.
The repair requires the orchestrator to know that the query is about a date, and to pick out the documentation in drive on that date, together with protecting a outdated doc that was reside on the time.
5.5 Rationale loss
Coated above. The reasoning lives in an e-mail, the rule lives in a suggestion, whereas the connection between them lives nowhere.
5.6 Multi-hop
“Why was this declare triaged Degree 1?” requires the declare notes, then the triage guideline, then the water-damage idea, then the coverage clause. 4 hops. High-k similarity search doesn’t traverse, it ranks. Typed relationships do traverse.
Put collectively, these six are the argument — not that the wiki is nicer, however that there’s a class of query retrieval solutions confidently and wrongly, and the information layer catches it.
6. Writing information is a special danger class than answering
Right here is the factor that took me longest to internalize, and it modified how I take into consideration the entire design.
A mistaken chat reply impacts one dialog. A mistaken canonical idea web page impacts each reply that’s later constructed on prime of it, for so long as it stays mistaken, and no one notices, as a result of it appears like information.
The second your system begins writing persistent information, it has crossed from “retrieval utility” into “system of report”, and it wants the controls that include that.
Every little thing is a patch
The mannequin by no means writes to the shop. It proposes a patch. The appliance validates it and, the place the change is consequential, a human approves it.
Be aware the place “resolve a contradiction” sits: by no means computerized. If the system may resolve contradictions by itself authority, the contradiction object can be pointless.
Provenance is a series, not a discipline
An object whose sources can’t be recognized must be deleted, not corrected. You can’t repair one thing once you don’t know the place it got here from.
Staleness is a property you need to observe
A web page may be appropriate on Monday and mistaken on Friday as a result of a supply was outdated beneath it. So each derived object carries last_validated_at, and the supply it was derived from carries superseded_by. When a supply is outdated, every part derived from it’s marked stale and should not be offered as present till it’s re-derived.
7. Routing: which layer solutions this?
Not each query wants each layers, and sending every part to each is the way you construct one thing costly and gradual.
Two issues about this diagram are deliberate.
First, the temporal verify belongs earlier than retrieval, not after. When you rank by similarity first and discard inapplicable outcomes afterwards, the inapplicable paperwork have already consumed your top-k. In manufacturing, put the efficient and outdated dates on the index and filter within the question itself, so the candidate set is constrained earlier than rating. The demo takes a shortcut right here that I ought to come clean with: it applies the date filter instantly after retrieval, which behaves identically at this corpus dimension however would quietly starve top-k on a big one. The precept stands that the demo trades it for a less complicated index schema.
Second, the contradiction verify is a gate on the way in which out. It doesn’t matter which route the query took. If the subject is contested, the system stops. In code, that’s roughly:
def question(self, query, requested_mode, top_k, as_of=None):
mode = self.choose_mode(query, requested_mode)
wiki_items = self._search_wiki(query, top_k) if mode in {"wiki", "hybrid"} else []
proof = self.proof.search(query, top_k) if mode in {"proof", "hybrid"} else []
if as_of:
# The date the query is ABOUT - not the date it's requested.
proof = self._filter_by_date(proof, as_of)
# Pull in each contradiction touching a retrieved idea, even when the
# contradiction object itself didn't rank. Somebody asking about hint and
# entry will get the battle whether or not or not they used the phrase "contradiction".
contradictions = self._contradictions_for(wiki_items)
warnings = self._warnings(proof, contradictions, as_of)
context = self._build_context(wiki_items, proof, contradictions, as_of)
reply = self.mannequin.reply(query, context)
... And the instruction that goes to the mannequin is unambiguous:
UNRESOLVED CONTRADICTIONS. You MUST current each positions with their sources and state that the place is unresolved. You MUST NOT select between them, and also you MUST NOT desire the newer doc - recency will not be applicability.
8. When it is best to not construct this
I might relatively you skip this structure than misapply it, so let me be direct concerning the circumstances the place easy RAG is the higher engineering alternative.
Stick with plain RAG when:
- the corpus is small and queried not often, so there may be nothing to amortize;
- customers overwhelmingly need precise supply lookup, not synthesis;
- paperwork churn so quick that any derived synthesis is stale earlier than it’s used;
- there is no such thing as a cross-session information price preserving;
- it’s a prototype with a brief life;
- ingestion latency has to be minimal;
- your group can not but govern AI-generated persistent information. This one will not be a technical constraint and it’s the one individuals ignore.
Construct the information layer when:
- the identical area is queried repeatedly, by individuals whose work continues throughout periods;
- cross-source synthesis is regular, not distinctive;
- selections and their rationale should survive workers turnover;
- exceptions, scopes and contradictions truly matter;
- area consultants must see and proper what the system believes;
- an audit path from reply to supply is a requirement, not a nice-to-have.
It is a workload determination relatively than a matter of new good, outdated dangerous, and the trustworthy reply for lots of functions is that you do not want this.
Half II – Implementing it on Azure
Every little thing above is intentionally moveable. Now let me construct it correctly on Azure, the platform I work with each day.
9. Mapping the layers to providers
| Accountability | Azure service |
| Immutable uncooked sources | Azure Blob Storage |
| Scanned PDFs, tables, varieties, structure | Azure AI Doc Intelligence |
| Chunk, key phrase, vector and hybrid retrieval | Azure AI Search |
| Chat and embedding mannequin deployments | Microsoft Foundry |
| Structured wiki state | Azure Cosmos DB for NoSQL |
| API and orchestration | FastAPI on Azure Container Apps |
| Occasion-driven ingestion | Occasion Grid → Container Apps Jobs |
| Id and secrets and techniques | Entra ID, managed identification, Key Vault |
| Telemetry | Utility Insights |
| Human inspection of the information | Obsidian, over exported Markdown |
10. Blob Storage – the one factor you can’t regenerate
Every little thing else on this structure is derived. Chunks may be re-chunked. Embeddings may be re-embedded. Your complete wiki can, in precept, be recompiled from scratch. The unique paperwork can’t be recovered from something.
So they get handled accordingly:
raw-sources/
{workspace-id}/
{document-id}/
original-file.pdf ← by no means rewritten
extracted-content.json ← Doc Intelligence output
ingestion-metadata.json ← what ran, when, which mannequin model
wiki-export/
{workspace-id}/
Residence.md
Ideas/ · Choices/ · Contradictions/ · Sources/
In Bicep, the half that issues is three properties:
useful resource blobService 'Microsoft.Storage/storageAccounts/blobServices@2023-05-01' = {
mum or dad: storage
title: 'default'
properties: {
// Originals should survive an ingestion bug. Versioning and delicate delete are the
// least expensive insurance coverage accessible on the one artifact the system can not regenerate.
isVersioningEnabled: true
deleteRetentionPolicy: { enabled: true, days: 30 }
containerDeleteRetentionPolicy: { enabled: true, days: 30 }
}
}
And on the storage account itself, one line that I might argue for in any manufacturing deployment:
allowSharedKeyAccess: false // no connection strings, everThe wiki-writing course of mustn’t ever be capable to contact the originals. That may be a provenance, audit and deletion-workflow requirement, and it’s a lot simpler to implement with separate containers and slim function assignments than with good intentions.
11. Doc Intelligence – used selectively
My demo corpus is .txt and .md, so the applying reads it instantly. Actual insurance coverage paperwork are scans, varieties, tables and signatures, and the structure itself is usually load-bearing. A desk of endorsement limits flattened right into a paragraph is worse than ineffective.
Azure AI Doc Intelligence offers you prebuilt and customized fashions that return textual content, tables, choice marks and construction.[3] My rule of thumb:
- clear textual content and Markdown → a easy parser, no cost;
- machine-readable PDFs → a PDF parser, whether it is genuinely ample;
- scans, varieties, advanced structure, tables → Doc Intelligence;
- at all times persist the extracted JSON subsequent to the authentic and maintain the web page and span offsets so a quotation can level at a location, not only a doc.
That final level pays for itself the primary time a compliance reviewer asks “the place precisely does it say that?”
12. Azure AI Search – the proof layer
Every listed report carries each searchable textual content and its vector:
{
"id": "INS-SYN-004-0",
"workspace_id": "ostermere-insurance-demo",
"source_id": "INS-SYN-004",
"title": "Excessive-Wind Zone Underwriting Replace",
"content material": "For brand spanking new Hearthmere enterprise in zone H3...",
"chunk_number": 0,
"content_vector": [0.012, -0.008, 0.031, "..."]
} Azure AI Search suits this design nicely for one particular cause: textual content and vector fields coexist in a single index, and a hybrid question runs the full-text and vector queries in parallel, fusing the rankings with Reciprocal Rank Fusion. Semantic rating can then reorder the highest outcomes.[4]
That issues greater than it’d sound. In insurance coverage, half the queries are conceptual (“what counts as sudden water injury”) and half are lexical (“what does HS-TA-01 cowl”). Vector search is sweet at first and unreliable at second: embeddings are identified to be weak with precise identifiers, clause numbers and product codes. BM25 handles these nicely however can not deal with paraphrasing. You need each, and also you need them fused relatively than chosen between.
from azure.search.paperwork.fashions import VectorizedQuery
vector_query = VectorizedQuery(
vector=self.mannequin.embed(question),
k_nearest_neighbors=max(top_k, 10),
fields="content_vector",
)
outcomes = self.consumer.search(
search_text=question, # BM25 leg
vector_queries=[vector_query], # vector leg
filter=f"workspace_id eq '{self.workspace_id}'", # safety trim
choose=["id", "source_id", "title", "content", "chunk_number"],
prime=top_k,
) Be aware the filter, as a result of it’s doing safety work, not relevance tuning.
⚠️ Vector similarity will not be authorization. Nothing about cosine distance respects your permission mannequin. Safety trimming should be a tough filter on an listed discipline, utilized at question time, on each single question. It should be utilized identically to the information layer and to the exported Markdown. A wiki web page that synthesizes three paperwork the person can not learn continues to be an information leak, only a properly formatted one.
For bigger programs, built-in vectorization can transfer chunking and embedding into the indexer pipeline.[5] I maintain the embedding calls within the utility right here purely, so the circulate is seen and explainable in an article.
One sizing be aware from truly deploying this: the demo runs on the free Search tier, and vector plus hybrid search work high quality there for a 21-document corpus. What the free tier offers up is the semantic ranker and managed-identity assist on the service itself, each are conditionals within the Bicep, so deal with fundamental as the ground for manufacturing and free as a wonderfully great way to validate the design for nothing.
13. Cosmos DB – the information layer
Cosmos DB for NoSQL holds the structured wiki. The entire design suits in three selections.
Partition key’s /workspace_id. Each wiki object carries a sort discriminator, so ideas, selections, contradictions and open questions all reside in a single container. Which means a single-partition question can pull a complete workspace’s information with no cross-partition fan-out, which is precisely the entry sample this method has.
partitionKey: {
paths: ['/workspace_id']
type: 'Hash'
} No graph database — but. Individuals attain for Gremlin or Neo4j the second they hear “relationships”. I might push it again. Express relationship information in a doc retailer deal with every part this method truly does: discover neighbours, comply with a typed edge, render an idea web page with its hyperlinks. That’s one or two hops.
A graph retailer earns its place when deep traversal is itself the workload: multi-hop affect evaluation, centrality, path-finding throughout a big community. If you’re not doing that, you’re paying for a second database and a second question language to keep away from writing WHERE c.source_id = @id.
Serverless, for now. Billing follows consumed request models, which fits a demo and a spiky early workload. Transfer to provisioned throughput upon getting measured actual RU consumption, not earlier than, and never as a result of the phrase serverless is trendy.[6]
Information-plane entry makes use of Cosmos’s personal RBAC system (SQL function assignments), which is separate from Azure RBAC and catches individuals out:
useful resource cosmosDataRole 'Microsoft.DocumentDB/databaseAccounts/sqlRoleAssignments@2024-11-15' = {
mum or dad: cosmos
title: guid(cosmos.id, identification.id, 'data-contributor')
properties: {
principalId: identification.properties.principalId
// 00000000-...-000000000002 is the built-in Cosmos DB Information Contributor
roleDefinitionId: '${cosmos.id}/sqlRoleDefinitions/00000000-0000-0000-0000-000000000002'
scope: cosmos.id
}
} Mixed with disableLocalAuth: true, there is no such thing as a key to leak.
14. Microsoft Foundry – fashions now, brokers later
Foundry offers the chat and embedding deployments. The appliance talks to it by means of the Azure OpenAI v1 interface utilizing the usual OpenAI Python SDK, which implies no api-version parameter to chase each few months:[7]
from openai import OpenAI
from azure.identification import DefaultAzureCredential, get_bearer_token_provider
credential = get_bearer_token_provider(
DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default"
)
consumer = OpenAI(base_url="https://YOUR-RESOURCE.openai.azure.com/openai/v1/", api_key=credential)
response = consumer.responses.create(mannequin="YOUR-CHAT-DEPLOYMENT", enter=immediate) For this demo, the FastAPI app orchestrates explicitly: embed, retrieve, search the wiki, construct context, name the mannequin, validate, write. I did that on objective since each step is seen, and you may put a breakpoint in any of them.
The pure evolution is Foundry Agent Service, the place the identical orchestration turns into an agent with a constrained toolset:[8]
search_evidence()
search_wiki()
get_concept()
check_contradictions() ← the gate, as a instrument
propose_wiki_patch() ← proposes solely; can not apply
apply_approved_patch() ← requires an approval token
export_obsidian_vault() I might not hand an agent unrestricted database entry on day one. Every instrument enforces its personal validation, authorization, logging and slim enter schema. Be aware that propose_wiki_patch and apply_approved_patch are separate instruments: the agent can attain the primary and can’t attain the second with no human in between. That separation is the entire governance mannequin, expressed as an API floor.
For extraction, use structured outputs, which constrain the response to a JSON Schema as an alternative of merely requesting legitimate JSON.[9] The distinction turns into apparent the primary time a manufacturing extraction returns almost-valid JSON.
I can report that from expertise now, as a result of deploying this stack produced two discipline notes price passing on:
- Truncated JSON, on the very first doc. gpt-5-mini is a reasoning mannequin, and its reasoning tokens are spent from the identical max_output_tokens price range as the reply. With a 5,000-token cap, the extraction JSON arrived reduce off mid-string. The repair within the repo: a 16,000-token price range, reasoning: {“effort”: “low”} for extraction work, JSON mode (textual content.format: json_object) to constrain the decoder, and one retry. A element that value me two failed seeding runs: the primary full seed succeeded with out JSON mode, then two later runs failed on completely different paperwork. Immediate-only JSON doesn’t fail reliably – it fails intermittently, which is worse. Schema-constrained structured outputs stay the actual repair; that is the pragmatic one.
- A deployment naming quirk. An embedding deployment named identically to its mannequin (text-embedding-3-small) got here up wholesome in each standing verify after which returned unknown_model on each single name, on each the v1 and the basic route. An an identical deployment named embed-3-small labored on the primary request. Chat deployments don’t present the issue. The Bicep now retains the deployment title and the mannequin title as separate parameters, with this story within the description.
15. FastAPI on Container Apps
The API floor:
GET /well being
GET /wiki checklist objects, filterable by sort
GET /wiki/contradictions the contradiction register
GET /wiki/{item_id}
POST /question { query, mode, top_k, as_of }
POST /ingest/textual content
POST /ingest/file
POST /cost-estimate
POST /export/obsidian
POST /export/obsidian.zip One implementation element that value me some debugging time and is price passing on: /wiki/contradictions should be declared earlier than /wiki/{item_id}, or FastAPI matches the trail parameter first and cheerfully appears for a wiki object with the id “contradictions”. Route order is important.
Container Apps is the suitable runtime right here as a result of it’s container-based with out asking me to run Kubernetes.[10] The scaling alternative is price a phrase:
scale: {
// For manufacturing, 1 heat duplicate avoids chilly begins; for a demo, 0 prices nothing.
minReplicas: minReplicas
maxReplicas: 3
} minReplicas is a parameter (default 1) as a result of the suitable reply is dependent upon what you’re operating. In manufacturing, scale-to-zero appears like free cash after which expenses you a chilly begin plus a mannequin handshake on the primary request after each scale-in. For a demo that’s exercised from a developer machine, zero is precisely proper — the validated deployment on this article ran at zero and value nothing whereas idle.
16. Id – no keys, anyplace
The entire deployment runs on a user-assigned managed identification with narrowly scoped roles, and each service has native auth disabled. The Container App will get AZURE_CLIENT_ID in its surroundings, DefaultAzureCredential picks it up, and no secret is ever issued to the applying.
{ title: 'AZURE_CLIENT_ID', worth: identification.properties.clientId } This issues extra on this structure than in a plain RAG app, and it’s price spelling out why. A retrieval system reads. This method writes persistent information that different solutions can be constructed on. The blast radius of a compromised credential will not be “somebody learn your paperwork” – it’s “somebody edited what your group believes”. Deal with the write path accordingly.
The complete infra/principal.bicep within the repo provisions every part: storage with versioning, AI Search with disableLocalAuth, Cosmos with serverless and SQL function assignments, Foundry with each mannequin deployments, Key Vault, Log Analytics, Utility Insights, the Container Apps surroundings and the app itself, plus the six function assignments that wire them collectively – and, once you cross your personal deployerPrincipalId, a mirrored set to your person account, which is what lets the seeding script and the assessments run from a developer machine with no single key.
az group create -n rg-wikirag-demo -l swedencentral
az deployment group create -g rg-wikirag-demo -f infra/principal.bicep -p namePrefix=wikirag 17. The ingestion lifecycle
That element within the center is the one individuals miss. When the mannequin extracts ideas from a brand new doc, it should see what the wiki already is aware of. In any other case, it invents “Money Settlement Foundation” as a brand-new idea, with out figuring out that actual-cash-value already exists with that precise alias, and your information base quietly forks.
Entity decision within the demo:
def _resolve_concept(self, title, aliases):
"""Precise id → title → alias.
With out this, 'money settlement foundation', 'depreciated worth' and 'ACV' every turn into
their very own web page and the wiki fragments into synonyms. The demo stops at alias
matching. A manufacturing system provides embedding similarity and an LLM adjudication
step for the ambiguous center, which is the place the attention-grabbing failures reside.
"""
candidates = {title.decrease(), *(a.decrease() for a in aliases)}
for idea in self.repository.list_items("idea"):
if idea["id"] == slugify(title):
return idea
identified = {idea["title"].decrease(), *(a.decrease() for a in idea.get("aliases", []))}
if candidates & identified:
return idea
return None The demo ships with a passing check that ingests a doc utilizing the phrase “depreciated worth” and asserts that no new idea web page is created, however it resolves onto the present actual-cash-value.
And here’s what occurs with out the stronger decision steps, measured relatively than argued. Once I ran all twenty-one paperwork by means of reside gpt-5-mini extraction on the deployed stack, the mannequin proposed ideas that alias matching may resolve solely partially. The end result was 149 idea objects the place the curated graph has 19 – a roughly 7x fragmentation issue. The machine-extracted ideas usually are not mistaken, they’re simply named in methods no alias checklist anticipated (“Roof Inspection Requirement”, “EUR 5,000 authorization restrict”). That quantity is the concrete argument for embedding similarity and LLM adjudication within the decision chain.
Half III – The demonstration
18. Ostermere Mutual
The artificial corpus is twenty-one paperwork for an imaginary property insurer. It’s sufficiently small to learn in a day and intentionally engineered in order that the six failure modes from stage 5 are all reside in it.
| Class | Paperwork |
| Coverage & product | Coverage overview v1.2, outdated v1.1, Part 4 wording (escape of water), endorsement catalogue |
| Underwriting | Roof guideline (20 yrs), H3 replace (15 yrs), wind-zone register, referral matrix, surveyor panel be aware |
| Claims | Triage guideline, claims guide ch.7, three declare recordsdata (CLM-1042, CLM-1108, CLM-1155) |
| Regulatory & compliance | Veyland round (fictional regulator), truthful claims dealing with commonplace, information provenance commonplace |
| Casual | Two e-mail threads, working-group assembly minutes, buyer FAQ |
These casual sources usually are not ornament. The assembly minutes are the place the contradiction is formally recorded as unresolved. The e-mail thread is the place the solely clarification of the 15-year threshold exists. In my expertise, that is precisely how actual organizations work; the principles are within the paperwork, and the explanations are in somebody’s inbox.
A be aware on knowledge provenance: each doc within the corpus is artificial and written by me (polished with AI) for this text, so there aren’t any licensing constraints on its use. The dataset ships with the repository beneath the identical MIT license because the code.
Compiled, that corpus produces:
21 sources · 19 ideas · 28 typed relationships · 4 comparisons · 5 selections · 2 contradictions · 4 open questions · 2 processes
19. Walkthrough one – the scoped rule
curl -X POST http://localhost:8000/question -H 'Content material-Sort: utility/json' -d '{
"query": "What's the roof inspection threshold?",
"mode": "hybrid"
}'
Plain retrieval finds the H3 replace, which says 15 years, and says “15 years”.
The information layer returns the choice object:
{
"id": "h3-roof-inspection-threshold",
"sort": "determination",
"abstract": "For brand spanking new Hearthmere enterprise in zone H3, request a roof inspection when the
main roof overlaying is greater than 15 years outdated. Outdoors H3, the overall
threshold of greater than 20 years stays in drive...",
"scope": "New enterprise solely. Zone H3 solely. Doesn't apply to in-force insurance policies
written earlier than the zone mannequin existed.",
"rationale": "Between 15 and 20 years, the frequency of whole or near-total overlaying
displacement within the severe-wind band runs at roughly 3.4x the usual
band on the similar roof age...",
"rationale_source": "INS-SYN-018",
"accountable_owner": "D. Lindqvist (Head of Property Underwriting)"
}The quantity by no means seems with out its scope. And the rationale, recovered from an e-mail, is correct there, which signifies that in eighteen months, when somebody asks why 15, the reply exists.
The e-mail’s creator, by the way, predicted this precise failure in writing:
“Please don't let this turn into a common 15-year rule. If somebody reads the replace with out the zone qualifier they may apply it to the entire ebook, we are going to demand inspections on tens of 1000's of completely odd roofs, and the dealer complaints can be totally justified.”
That may be a artificial quote I wrote to make a degree, however I don’t suppose it’s an unrealistic one.
20. Walkthrough two – the contradiction
That is the one I might put in entrance of a sceptical architect.
curl -X POST http://localhost:8000/question -H 'Content material-Sort: utility/json' -d '{
"query": "Is hint and entry coated beneath Hearthmere?",
"mode": "wiki"
}' A retrieval system solutions this. It retrieves a bit (from the claims guide, or from the endorsement catalogue) and tells you both “sure, as much as €5,000 as commonplace” or “provided that you obtain HS-TA-01”. Each solutions are supported by an actual doc. Each are mistaken, as a result of the agency doesn’t have a place.
The hybrid system responds:
{
"reply": "The place is NOT SETTLED. The information base holds an unresolved
contradiction overlaying this query, so no reply is given...",
"warnings": [
"UNRESOLVED CONTRADICTION (con-001): Trace and Access - Standard Cover or Paid
Endorsement? The system will not choose between the conflicting sources.
Owner: Y. Tanaka (Product)."
],
"contradictions": [{
"id": "con-001",
"status": "unresolved",
"accountable_owner": "Y. Tanaka (Product)",
"statements": [
{ "source_id": "INS-SYN-012", "locator": "Chapter 7.3",
"effective_date": "2025-11-01",
"statement": "Trace-and-access costs are covered as standard under Hearthmere,
up to EUR 5,000..." },
{ "source_id": "INS-SYN-008", "locator": "HS-TA-01",
"effective_date": "2026-01-01",
"statement": "Trace and access is an optional endorsement (HS-TA-01)...
Where it does not appear on the policy schedule, trace-and-access
costs are not payable." }
],
"why_not_resolved": "Each paperwork are present. Neither supersedes the opposite. They
have been written by completely different groups. The Endorsement Catalogue is the newer
doc, however recency will not be applicability..."
}]
} Each statements. Each sources. Each dates. An accountable human being. And no reply, as a result of there may be no one.
That JSON is demo mode’s deterministic scaffold. Here’s what the deployed stack does with the identical query – gpt-5-mini, reside, over Cosmos DB and AI Search:
There’s an UNRESOLVED CONTRADICTION related to this query. Don’t deal with the difficulty as settled.
- INS-SYN-012 (Chapter 7.3, efficient 2025-11-01): “Hint-and-access prices are coated as commonplace beneath Hearthmere, as much as EUR 5,000. Handlers ought to authorize affordable expenditure as much as this restrict with out referral.”
- INS-SYN-008 (HS-TA-01, efficient 2026-01-01): “Hint and entry is an elective endorsement (HS-TA-01), restrict EUR 5,000, indicative premium EUR 24. The place it doesn’t seem on the coverage schedule, trace-and-access prices usually are not payable.”
This contradiction is UNRESOLVED. Accountable proprietor: Y. Tanaka (Product) [con-001].
I can not resolve or state whether or not trace-and-access is roofed beneath Hearthmere due to the unresolved battle above.
The mannequin was not requested to be cautious on the whole. It was handed the contradiction object and one rule, and it adopted the rule.
The second-order impact is the half I discover genuinely attention-grabbing. As soon as a contradiction is a saved object, different issues can rely on it. The corpus incorporates a policyholder query on declare CLM-1155 (is the leak-detection survey value payable?) which can’t be answered till this battle is resolved. So, it’s saved as an open query with blocked_by: “con-001”.
The system is aware of why it can not reply, and it is aware of who has to resolve earlier than it could possibly.
21. Walkthrough three – the date of loss
Declare CLM-1108: storm injury, roof overlaying displaced, date of loss 20 February 2026. The roof is eighteen years outdated. The property, beneath at this time’s classification, can be in zone H3.
Ask a retrieval system whether or not an inspection was required, and it’ll discover the H3 rule (15 years, and 18 > 15) and inform you the roof was over threshold, and no inspection was on file. It appears like a discovering. It’s the starting of a wrongful declinature.
As a result of the H3 rule took impact on 1 March 2026. 9 days after the loss. And the coverage was written in September 2025, earlier than the zone mannequin existed in any respect, so the property carried no zone classification on the day the roof got here off.
curl -X POST http://localhost:8000/question -H 'Content material-Sort: utility/json' -d '{
"query": "What roof inspection threshold utilized to this property?",
"mode": "proof",
"as_of": "2026-02-20"
}' The as_of parameter is the date the query is about – not the date it’s requested. With it set, INS-SYN-004 is excluded from the candidate set totally. It was not in drive. The relevant threshold was the overall 20-year rule, the roof was 18 years outdated, it was beneath threshold, and no requirement was breached.
The identical mechanism runs within the different path. Ask about settlement foundation with as_of: “2025-08-01” and the system retrieves the outdated coverage v1.1, as a result of that’s the model that ruled a loss on that date, and marks it:
{
"source_id": "INS-SYN-009",
"in_force_at_as_of": true,
"be aware": "outdated at this time, however in drive on 2025-08-01 - that is the model that
governs a loss on that date"
} Underneath v1.1, depreciation was not recoverable. Underneath at this time’s v1.2, it’s. Identical time period, similar coverage, reverse reply – and the one factor that distinguishes them is a date that similarity search can not see.
Each behaviours are coated by assessments within the repo, and each reproduce on the reside deployment. At as_of: 2026-02-20 the deployed mannequin answered:
Brief reply: The overall >20-year inspection threshold utilized. [INS-SYN-003]
Why: On 2026-02-20 the Hearthmere guideline required an inspection when the first roof overlaying is greater than 20 years outdated [INS-SYN-003]. The H3 15-year threshold … didn’t take impact till 1 March 2026; this coverage was written 14 Sep 2025 (so the H3 rule didn’t apply to it) and the roof was ~18 years outdated at loss. [INS-SYN-018] [INS-SYN-013]
And at as_of: 2025-08-01:
- Settlement foundation: Precise Money Worth (ACV) – outlined as substitute value much less depreciation – for eligible constructing claims beneath Ostermere Hearthmere v1.1. [INS-SYN-009]
- Is depreciation recoverable? No. Underneath v1.1 depreciation will not be recoverable after restore. [INS-SYN-009]
They’re the 2 assessments I might maintain if I needed to delete each different one.
Another, reside and unscripted: the multi-hop query
Part 5.6 claimed that typed relationships let the system traverse the place similarity search solely ranks. I had not scripted a demo for it, so I requested the deployed stack instantly: “Clarify why the CLM-1042 water loss was positioned at triage Degree 1 beneath the property declare triage course of.” The reply walked the entire chain:
- Governing Degree-1 standards: Degree 1 – Commonplace is outlined as “Minor seen injury, no security concern, no persevering with water entry, and estimated publicity beneath EUR 5,000” [INS-SYN-002].
- How CLM-1042 matches these standards: property remained liveable, no reported electrical fault [INS-SYN-005]; water circulate stopped inside roughly 40 minutes [INS-SYN-005]; injury confined to the water path [INS-SYN-005; INS-SYN-014]; preliminary restore estimate EUR 3,800, beneath EUR 5,000 [INS-SYN-005; INS-SYN-002].
- Supporting operational steering: the Claims Dealing with Guide lists indicators of a sudden escape of water … which align with CLM-1042’s info [INS-SYN-012]. CLM-1155 explicitly contrasts CLM-1042 as a datable, sudden failure [INS-SYN-014].
- Be aware on provisional standing: protection and exclusions stay topic to a proper protection dedication by a certified claims skilled [INS-SYN-005].
4 sources, 4 hops, and the human-decision boundary preserved on the finish with out being requested for.
One trustworthy remark from the identical session: my first phrasing of that query (“…and the way does the water injury cowl apply to it?”) acquired gated by the contradiction – CLM-1042’s proof touches the contested trace-and-access idea, so the system offered con-001 and declined to make a settled dedication as an alternative of answering the triage half. The gate errs on the facet of stopping. For a claims workflow I think about that the suitable default, however it’s a actual trade-off: an aggressive gate will generally maintain again a solution the person legitimately wanted and tuning that boundary is a part of working the system.
22. Seeing it: the Obsidian vault
The appliance exports all the Cosmos DB state right into a vault. Fifty-five pages, all generated, nothing hand-written:
obsidian_vault/
Residence.md
Open Questions.md
Ideas/ 19 pages
Sources/ 21 pages
Choices/ 5 pages
Comparisons/ 4 pages
Contradictions/ 2 pages ← con-001, con-002
Processes/ 2 pages 
Open the folder in Obsidian, and you may navigate hyperlinks, examine backlinks, comply with the graph, spot orphan pages, and, most significantly, see what the system believes and inform it that it’s mistaken.
That final functionality is, I feel, the strongest argument for this whole structure. A vector index is operationally wonderful and utterly opaque to a website knowledgeable. You can’t hand an underwriter a 1,536-dimension embedding and ask, “does this look proper to you?” You can hand them a Markdown web page that claims the brink is 15 years, however solely in H3, and just for new enterprise, and right here is why, and listed here are the 4 paperwork it got here from.
They may inform you inside thirty seconds whether or not it’s proper. Markdown makes the reminiscence auditable by the individuals who truly know the area. No different a part of the stack does that.
23. The price argument, actually
The information layer prices extra at ingestion. I’m not going to faux in any other case.
A RAG pipeline extracts and embeds every doc as soon as. The hybrid pipeline moreover summarizes, extracts ideas and claims, resolves entities in opposition to the present wiki, generates relationship and comparability patches, validates, and regenerates Markdown. The write amplification is actual, and it’s not small.
The argument is that this front-loaded value buys down repeated query-time reasoning. So, the query will not be whether or not the wiki prices extra to construct (it plainly does) however whether or not the compilation is amortized throughout sufficient future use.
The mannequin in cost_model/cost_model.py:
Compilation = D × Td × M
Easy RAG = Q × Tr
Hybrid = Q × (Tw + V × Television) With the illustrative defaults – 50 paperwork, 6,000 tokens every, 1,000 questions, 6,000 retrieved context tokens per RAG query versus 1,350 wiki context tokens, uncooked verification on 25% of questions:
| Measure | Tokens |
| Supply corpus | 300,000 |
| Wiki compilation | 501,000 |
| Easy RAG, 1,000 questions | 6,000,000 |
| Hybrid, 1,000 questions | 1,725,000 |
| Context saved | 4,275,000 |
| Break-even | ~117 questions |
Now the caveats, as a result of a chart like this will simply mislead:
- That is token quantity, not worth. It ignores output tokens, embedding prices, AI Search capability, Cosmos RUs, Container Apps compute, and Doc Intelligence pages.
- It treats each token as equal. In apply, ingestion and answering can use completely different mannequin lessons, and that’s the place a lot of the actual saving lives, as a result of a small mannequin can reply from concise wiki context whereas a stronger one is reserved for reconciliation and updates.
- It doesn’t assure something. A badly ruled agent that rewrites the entire wiki on each ingestion will erase any saving you modelled. The economics rely totally on disciplined replace insurance policies.
One measured knowledge level, from deploying this precise stack: ingesting all twenty-one paperwork by means of reside gpt-5-mini extraction and embeddings, operating each walkthrough on this article, and regenerating the vault from Cosmos DB value roughly $0.20-0.30 in whole. The idle stack – free-tier search, scale-to-zero Container App, serverless Cosmos – burns about $0.05 a day. At this corpus dimension the compilation value is espresso cash. The economics solely turn into attention-grabbing at scale, which is what the mannequin above is for.
The context-size assumption additionally survived contact with the deployed system. Measured throughout a set of conceptual questions on the reside stack, the wiki context averaged roughly 500 tokens in opposition to roughly 1,750 for the equal proof context – a 3.5x ratio, in the identical vary because the 4.4x the mannequin assumes. Absolutely the numbers are smaller than the mannequin’s, as a result of the artificial paperwork are quick. The ratio is the half that transfers to an actual corpus. If something, the measured ratio is barely extra conservative than the assumed one, which strikes the break-even later, not earlier — price figuring out earlier than you quote the mannequin at a price range assembly.
Run /cost-estimate with your personal assumptions. Then throw them away and use telemetry out of your precise corpus and question combine, priced with the present Azure calculator.[11]
And actually, the token argument is the weakest argument for this structure. The actual returns are:
- constant terminology throughout periods and throughout individuals;
- selections and rationale that survive the one that made them depart;
- contradictions which might be seen as an alternative of silently resolved;
- an audit path from any reply to its supply;
- a information artifact a website knowledgeable can evaluation;
- continuity throughout agent periods and throughout mannequin upgrades.
I might take over just a few million enter tokens.
24. Governance, restated
As a result of the system writes, it wants to manage a read-only system doesn’t.
Protect provenance. Each assertion traces to a bit, to a span, to a doc, on the model it was derived from.
Patch, by no means write. The mannequin proposes; deterministic validation and, for something consequential, a human disposes.
Detect staleness. last_validated_at, superseded_by, source_effective_date. When a supply is outdated, every part derived from it’s stale till re-derived.
Trim safety at each layer. Blob paths, search information, Cosmos objects, Markdown exports, agent instruments, caches, telemetry. Persistently. A synthesized web page mustn’t ever floor data the reader couldn’t entry within the supply.
Maintain the human determination human. Within the demo, the AI could summarize an consumption, retrieve coverage proof, determine lacking data, suggest a provisional triage degree and flag conflicting guidelines. It could not decide protection, decline a declare, assess fraud, fee a danger, resolve a contradiction, or inform a buyer a call has been made.
The artificial working-group minutes include the perfect articulation of this that I managed to jot down, and I’ll let it stand because the governance precept for the entire structure:
Handlers at the moment undertake the assistant’s proposed triage degree in roughly 90% of circumstances, which is ok, however solely as a result of they’re studying the consumption themselves. Eradicating the handler from the loop removes the factor that makes the 90% reliable.
That’s the lure in a single sentence. An automatic system earns credibility beneath human evaluation, after which that credibility is used because the argument for eradicating the evaluation.
25. What I might construct subsequent
The repo is a baseline, not a product. The gaps I’m most aware of:
- Occasion Grid and queue-driven async ingestion (the demo ingests synchronously as a result of it’s simpler to run regionally).
- Doc Intelligence with web page and span preservation, so citations level at places.
- Strict JSON Schema structured outputs on each extraction and patch.
- Entity decision with embedding similarity and LLM adjudication, not simply alias matching.
- A human approval UI for high-risk patches: proper now the lifecycle exists within the design and the low-risk path exists within the code.
- Foundry Agent Service instruments, with suggest and apply as separately-permissioned surfaces.
- Analysis units for retrieval, synthesis and the onerous one: replace accuracy. How do you check {that a} information base modified appropriately?
- Freshness and contradiction dashboards. A contradiction register no one appears at is only a log file.
- Per-tenant safety trimming, finish to finish.
- Mannequin routing by activity complexity and danger.
Quantity 7 is the genuinely open analysis drawback, and I should not have a superb reply to it but.
26. To sum all of it up
RAG is the proof engine of this structure, and nothing right here is its obituary. It offers the mannequin entry to authentic, related, present supply materials, and there’s no substitute for that.
What it doesn’t do is bear in mind.
The information layer provides the factor that was lacking: a maintained, structured, inspectable illustration of what the system has already labored out, with the scopes intact, the rationale preserved, the contradictions seen, and a line again to the proof for each declare.
RAG asks: What ought to I retrieve for this query?
The information layer: What must be durably true after processing every part to this point?
The orchestrator: Which of these do I must reply this safely, proper now?
On Azure that separation maps cleanly:
- Blob Storage preserves the originals — the one factor you can’t regenerate.
- Doc Intelligence extracts the tough content material and retains the spans.
- Azure AI Search shops retrievable, security-trimmed proof.
- Cosmos DB shops the evolving ideas, relationships, selections and contradictions.
- Microsoft Foundry offers the fashions, and the trail to brokers.
- FastAPI on Container Apps runs the orchestration, the temporal scoping and the contradiction gate.
- Obsidian makes the entire thing seen to the individuals who know whether or not it’s proper.
It’s extra work than easy RAG, and it prices extra to ingest. In change you get organizational reminiscence, constant synthesis, express relationships, a visual determination historical past, and, the half I maintain coming again to, a system that can inform you “two of our paperwork disagree and no one has determined but” as an alternative of confidently making one thing up.
That final functionality will not be a characteristic. It’s the cause to construct it.
The appliance is not looking a pile of paperwork. It’s slowly constructing a reviewable mannequin of a website, whereas protecting the unique proof shut sufficient to verify each necessary conclusion in opposition to.
Thanks for taking the time to discover this structure with me. It was an extended piece than I supposed, as a result of the design stored having yet another half price explaining. The FastAPI venture, the Bicep templates, the twenty-one artificial paperwork and the Obsidian vault are all in the repository, and I strongly imagine they offer you a sensible start line for constructing a persistent information layer of your personal. Clone it, run it in demo mode with none Azure credentials, and attempt to break it — I might genuinely like to listen to the place it fails.
Disclosure: I’m a Microsoft MVP. This text displays my very own impartial work and opinions; Microsoft had no involvement in or evaluation of its content material. All Azure utilization described relies on public documentation and my very own deployment.
References
[1] P. Lewis et al., Retrieval-Augmented Era for Information-Intensive NLP Duties (2020), NeurIPS 2020
[2] A. Karpathy, LLM Wiki (2025), GitHub Gist
[3] Microsoft, Doc Intelligence Format Mannequin (2026), Microsoft Study
[4] Microsoft, Hybrid Search Overview – Azure AI Search (2026), Microsoft Study
[5] Microsoft, Built-in Vectorization in Azure AI Search (2026), Microsoft Study
[6] Microsoft, Azure Cosmos DB Serverless (2026), Microsoft Study
[7] Microsoft, Azure OpenAI v1 API Lifecycle (2026), Microsoft Study
[8] Microsoft, Foundry Agent Service Overview (2026), Microsoft Study
[9] Microsoft, Structured Outputs with Azure OpenAI (2026), Microsoft Study
[10] Microsoft, Deploy a Flask or FastAPI Internet App on Azure Container Apps (2026), Microsoft Study
[11] Microsoft, Plan and Handle Prices of Azure AI Search (2026), Microsoft Study
