Essentially the most helpful reply a RAG system can provide is usually “that isn’t on this doc.” However when requested a query, a mannequin tends to reply anyway, so the trustworthy “not discovered” must be in-built, not hoped for. Getting the system to say no, and to be proper when it does, is more durable than getting it to say sure.
This text is a bonus in Enterprise Doc Intelligence, a sequence that builds an enterprise RAG system from 4 bricks. It justifies the “I don’t know”: a assured improper reply is a bug, a naked no-answer is nearly as dangerous, and every of the 4 bricks has one piece of proof to indicate.
🧭 New to the sequence? Each article on this sequence sits on our two In the direction of Information Science writer pages, Angela Shi and Kezhan Shi. That’s the shortest technique to see what is roofed and the place this one sits.
📓 Runnable companion notebooks are on GitHub: doc-intel/notebooks-vol1.
Ask the company chatbot “How a lot electrical energy does AI eat globally?” and it returns “I can not discover a solution on this doc.” Rephrase and ask once more: similar reply. At that time most customers surrender and go to Google.
The system did the correct factor. The doc it runs on is the World Financial institution’s Commodity Markets Outlook (April 2025 problem; CC BY 3.0 IGO, as declared on the World Financial institution Open Information Repository phrases of use which states the default license for pre-2023 publications), a 63-page quarterly report on oil, agriculture, and metals costs. AI electrical energy consumption will not be in there. There isn’t a reply to search out. The system was not mendacity.
However the consumer didn’t be taught something helpful. They can not inform whether or not the pipeline seemed all over the place it ought to have, or whether or not it gave up after one shallow embedding search. “I can not discover a solution” is a verdict that should include a protection. In any other case it reads as failure, not as a truth.
The sequence argued that structured output (Article 8) makes “sure” solutions verifiable, by forcing the mannequin to quote its proof: a quote, a web page quantity, a confidence. The identical logic applies to “no” solutions, and the identical 4 bricks contribute, one piece of proof every. This text walks the 4 bricks as soon as extra, on a query whose proper output is no, and reveals what every brick has to provide so the no-answer is defensible.
1. The 4 bricks, 4 sorts of proof
The argument compresses to at least one desk.
The remainder of the article walks every row so as, then ties them collectively on the CMO / AI electricity-consumption case.
2. Parsing: the relational tables are the proof
The price of a lacking parse is uneven. A false constructive (extracting noise) makes retrieval messier however is recoverable downstream. A false adverse (failing to extract an actual token) is silent: the no-answer verdict will look right, however it’s improper. The reply was there; the pipeline simply by no means noticed it.
The excellent news is that the parsing brick (Article 5B, the relational information mannequin) already produces precisely the info the absence declare wants. The output of parse_pdf will not be a wall of textual content; it’s a small relational set of DataFrames: one per type of factor the doc incorporates. line_df is the central desk: one row per textual content line throughout the entire doc, with its web page quantity, bounding field on the web page, and the detected line sort (prose, heading, desk row, determine caption). The opposite tables dangle off it.
For a no-answer verdict, we don’t write a brand new parsing operate. We mixture over the tables that exist already:
The three locations an absence declare nonetheless must examine past line_df:
-
Pictures that carry textual content: Charts usually maintain their axis labels and legends as image-embedded textual content relatively than typographic spans. The
image_dfregistry lists each such picture; an OCR go provides anocr_textcolumn to that desk with out touchingline_df. The parsing brick exposes each earlier than and after states; the protection report names what number of photos had been OCR’d. -
Tables: A quantity in a cell parses as a line whose neighbours are the opposite cells of the identical row, not the phrases above it on the web page. If the reply is “AI consumed 460 TWh in 2024” and the determine sits in a desk with out the phrase AI in any close by cell, a sweep on AI misses it.
object_registryflags which pages host tables so the retrieval brick can do a column-aware sweep there (Article 5 part on desk parsing). -
Cross-references that didn’t resolve.
cross_ref_dflists each “see annex B”, “determine 3.D”, “desk 11.E” point out within the physique. Any row whose goal will not be inobject_registryis a parsing hole: the physique guarantees an object the extraction didn’t discover. Every hole is a candidate spot the place the reply may very well be hiding.
The parsing brick’s piece of proof is due to this fact a small abstract derived from the tables it already produces, not a brand new element. The no-answer verdict can state: “63 pages parsed, 63 with textual content, 0 unresolved cross-references.” That’s sufficient for the consumer to belief the parsing layer did its work.
3. Query parsing: enumerate the vocabulary
The retrieval step is simply pretty much as good because the key phrases it sweeps on. If the skilled lists AI however forgets synthetic intelligence, the retrieval misses each line that spells the idea out. The work of the question-parsing brick is to push the key phrase set to exhaustion, with the skilled because the supply of reality and an algorithmic security web beneath.
3.1 The skilled is aware of the vocabulary
In enterprise RAG, the skilled virtually all the time is aware of the key phrases of their area. That is the straightforward case and it covers most questions. They record the ideas that the reply would point out, and for every idea they enumerate the variants: synonyms, acronyms, abbreviations, English / French / German types if the corpus is multilingual.
For the CMO / AI electrical energy instance, the skilled gives this desk:
concept_keywords_df schema from Article 6 utilized to a ‘no reply’ query – Picture by writerThat is precisely the concept_keywords_df schema from Article 6. The identical information construction that drives regular retrieval additionally drives the absence declare. The signature is equivalent; solely the interpretation of the output modifications.
3.2 Idea clustering as a security web
The uncommon case is the skilled who isn’t positive they listed each variant. The security web is algorithmic: cluster the corpus tokens by embedding similarity, then expose the clusters that the skilled’s key phrases land in, and ask whether or not any cluster member must be added.
The output is an inventory, not a choice. The skilled decides what so as to add. The candidates are a suggestion to contemplate, not a alternative for the skilled’s judgment. The sequence’ editorial place holds: amplify the skilled, don’t change them. The clustering reveals what the skilled might not have considered; the skilled validates what belongs within the key phrase set.
The key phrase set, signed off by the skilled, is the question-parsing brick’s piece of proof.
4. Retrieval: sweep, not top-k
For a traditional query (Article 7), retrieval returns top-k: the few pages or strains most certainly to comprise the reply. For a no-answer declare, that framing fails. The system can not show the reply is absent by trying on the top-10 pages. It has to have a look at each web page that mentions any of the ideas.
The form of the retrieval name due to this fact modifications:
The return worth is a DataFrame, not a ranked record. Every row is an EVIDENCE entry: “on web page P, line L, the variant V of idea C was discovered within the snippet S”. The cardinality of the result’s what the no-answer verdict activates:
-
Zero rows for some idea (no variant seems anyplace) → sturdy proof the doc doesn’t cowl that idea in any respect.
-
A number of rows however by no means co-located (ideas seem on totally different pages, by no means collectively) → medium proof: the ideas exist however no passage talks about their intersection.
-
Co-located rows (a number of ideas in the identical web page or paragraph) → weak proof of absence. Technology must take a look at the snippets and determine whether or not they reply the query or solely contact it.
On the CMO April 2025 case, the result’s unambiguous. The total sweep on the key phrase set above returns:
The hit-list is the retrieval brick’s piece of proof. On this query and this doc, the decision writes itself.
5. Technology: structured “no reply” with justification
The final brick takes the parse protection report, the validated key phrase set, and the hit-list, and turns them into the precise response the consumer sees. The schema mirrors AnswerWithEvidence from Article 8, however the energetic fields are totally different.
The technology step doesn’t invent any of the fields. The schema’s first three fields come straight from the earlier bricks (parse protection, the key phrase set, the hit-list). The mannequin’s job is narrower: choose the rationale, select which mentions to flag as closest, write the one-sentence “why this isn’t a solution” for every, and optionally counsel a reformulation.
On the CMO / AI electrical energy case, the mannequin’s output seems like this:
A consumer studying this output learns three issues without delay. The system did search for AI beneath 9 totally different names and located zero hits anyplace in 63 pages. There may be one electricity-related passage, but it surely issues India’s coal-driven energy demand, not AI. And if they need the reply, the CMO is the improper corpus to ask. The no-answer is now a helpful response.
6. The place this stops
Three edge circumstances the framework above doesn’t cowl cleanly, so as of significance.
Partial solutions: A query might have a part of its reply within the corpus and half exterior it. “How does the EU regulate AI beneath the AI Act, and the way does that examine to U.S. coverage?” on an EU-only authorized corpus. The retrieval brick will return hits for the EU aspect and nothing for the U.S. aspect. The fitting response is neither a assured sure nor a clear no; it’s a structured partial reply that reveals what was discovered and is specific about what was not. The schema for that could be a third sibling of AnswerWithEvidence and AbsenceJustification, with each answer_partial: str and missing_concepts: record[ConceptSearch] fields.
Ambiguous questions: “What about protection?” requested with out context. The system can not sweep as a result of there isn’t any outlined idea set. The fitting response is a clarification request, not a no-answer. The sign that distinguishes the 2 is the parsing of the query (Article 6): if the query parser can not extract ideas, the query is the issue, not the corpus.
Hostile or out-of-scope questions: “What’s the that means of life?” on an insurance-policy corpus. The system can appropriately say no-answer, however spending effort on the parse-coverage report and the closest-mentions is wasted. The pipeline ought to have an upstream “is that this query in scope” examine that short-circuits the total sweep when the query parser flags the ideas as unrelated to any corpus tag. Pipeline value issues when this sort of question is frequent.
These circumstances share a construction: the no-answer schema above is the correct form for absence claims, however it isn’t the correct form for each “I can not assist” output. Treating it as considered one of three siblings (sure / partial / no) relatively than a single fallback handles the variation cleanly.
7. Conclusion
A defensible “no reply” will not be a single sentence; it’s a chain of proof the 4 bricks produce collectively. Parsing reviews protection, query parsing reviews the skilled key phrase set, retrieval reviews the sweep, technology reviews the closest point out and why it doesn’t reply. The consumer sees the work and might dispute the parse, the key phrase set, the closest point out.
The symmetry with the remainder of the sequence is the larger level. Sure solutions are verifiable when the schema forces the mannequin to quote its proof (Article 8); no solutions are verifiable when the schema forces the pipeline to reveal its search. It’s the similar self-discipline on the identical bricks, and each outputs land structured.
8. Sources and additional studying
The canonical benchmark establishing “no reply” as a first-class output is Rajpurkar et al. (SQuAD 2.0, ACL 2018). The framing the place many questions are unanswerable contained in the given passage is Choi et al. (QuAC, EMNLP 2018). The model-side reflection-token analogue of the pipeline-side sweep used right here is Asai et al. (Self-RAG, ICLR 2024). The article’s framing: the three-sibling reply schema (sure / partial / no) with a defensible proof chain on the no-answer department, parse-coverage report, expert-keyword sweep, closest-mentions, each step auditable.
Earlier within the sequence:
-
Doc Intelligence: sequence intro. What the sequence builds, brick by brick, and in what order.
What works, what breaks
-
Baseline Enterprise RAG, from PDF to highlighted reply. The four-brick pipeline finish to finish: PDF in, highlighted reply out.
-
Embeddings Aren’t Magic: The Predictable Failure Modes of RAG Retrieval. The place embedding similarity wins (synonyms, typos, paraphrase), the place it predictably breaks (unknown phrases, negation, term-vs-answer relevance), and easy methods to use it anyway.
-
RAG will not be machine studying, and the ML toolkit solves the improper drawback. Why chunk-size sweeps and finetuning optimize the improper factor; route by query sort as an alternative.
-
From regex to imaginative and prescient fashions: which RAG method matches which drawback. Two axes, doc complexity and query management, that choose the method for every case.
-
10 frequent RAG errors we maintain seeing in manufacturing. Ten manufacturing errors, organized brick by brick, with the repair for every.
-
Doc parsing
-
Constructing Doc Construction with Loop Engineering: Recovering a PDF’s Define from Physique Typography for RAG. Rebuilding the define from physique typography when the PDF ships no contents web page in any respect: six alerts, one bounded loop.
-
Earlier than Full Agentic RAG: Know How You Resolve, and the Parsing Strategies You Decide From. The parsing strategies as a listing, and the choice of which to run, earlier than handing the loop to an agent.
Technology
-
Most RAG Hallucinations Are Extraction Errors: Seven Patterns for a Typed Technology Contract. Seven recurring methods a mannequin will get the extraction improper, and the typed contract that catches each.
-
Loop engineering for RAG technology: an LLM cascade from an inexpensive native mannequin as much as a hosted flagship. Beginning on an inexpensive native mannequin and escalating solely when the reply doesn’t maintain up, measured.
One-document pipelines
-
Minimize an Enterprise RAG Pipeline’s Latency and Value by Calling the LLM Much less, Not by Shopping for a Quicker Mannequin. Reducing a pipeline’s latency and value by calling the mannequin much less usually and cheaper, not by shopping for a sooner one.
-
RAG workflow and loop engineering: the dispatcher that decides when to loop and when to cease. Suggestions loops, bounded iteration, and the dispatcher, composed into one workflow.
-
Loop engineering for RAG: the small loops inside every step, the large loops throughout the pipeline. The 2 scales of loop: small bounded loops inside every brick, massive generation-triggered loops throughout them.
-
One doc to a corpus
-
Three Sorts of RAG Corpus, and What It Prices to Construct for the Improper One. Three questions that let you know which form a doc assortment has, and the worth of constructing for the improper one.
