RAG Is Not Machine Studying, and the ML Toolkit Solves the Incorrect Drawback

Machine studying solves issues the place the true reply is unknown and needs to be predicted. Will this buyer churn? What’s the likelihood this transaction is fraud? Is that this picture a cat? You don’t know the reply prematurely. That’s why you practice a mannequin. The mannequin learns from labeled examples, generalizes to new inputs, and produces a prediction. Efficiency is measured in combination, throughout hundreds of check instances, as a result of particular person predictions might be flawed whereas the mannequin remains to be helpful general.

RAG solves a distinct drawback. The reply to “what’s the efficient date of this contract?” exists, written on web page one of many doc, or it doesn’t exist wherever. There’s nothing to foretell. The system both finds the reply within the doc and reviews it faithfully, or it fails and will say so. Efficiency is binary on the query stage (bought it or didn’t) even in case you measure combination charges throughout many questions.

These variations are concrete:

• In ML, “the mannequin was flawed on 8% of instances” is a characteristic of the system. You construct redundancy, downstream checks, human evaluate for the borderline instances. In RAG, “the system gave a flawed reply 8% of the time” is a bug. Every of these 8% has a particular trigger: the flawed passage was retrieved, the best passage was retrieved however the mannequin paraphrased it badly, the reply wasn’t within the corpus and the system made one up. They aren’t statistical noise to optimize on common. They’re individually fixable failures.
• In ML, you’ll be able to’t usually inform why the mannequin bought a specific case flawed. That’s why explainability is a analysis discipline. In RAG, you’ll be able to at all times inform. The retrieval logs which passages it returned. The generator noticed precisely these passages. If the reply is flawed, you stroll the chain backward and discover the damaged hyperlink. There’s nothing hidden.
• In ML, the mannequin improves by coaching on extra knowledge. In RAG, the system improves by indexing higher, parsing extra fastidiously, retrieving extra exactly, prompting extra clearly. None of that’s coaching. It’s engineering.

That distinction modifications which instruments you attain for when one thing breaks.

The instances catalogued in Article 2 fall precisely right here: negation, actual identifiers, inside acronyms, sign dilution in lengthy context, topical proximity outranking the precise reply. None of these transfer if you swap embedding fashions or sweep chunk sizes. They aren’t bugs a mannequin can study its method out of, as a result of there isn’t any labeled sign saying “that is the best line” for the mannequin to coach on. The repair is structural (query parsing, professional key phrases, retrieval that is aware of the doc’s construction), and the following sections stroll via the three ML reflexes that choose the flawed software as an alternative.

Three ML strategies get imported into RAG initiatives by default: hyperparameter optimization, analysis datasets with practice/check splits, and feature-attribution explainability. Every is cheap inside ML. Every misfires right here.

The most typical framing goes one thing like this: chunk measurement, overlap, top-k, similarity threshold. These are hyperparameters, and it’s best to optimize them the best way you optimize ML fashions, utilizing instruments like Optuna or Ray Tune. Run a sweep, plot the curves, choose the most effective configuration.

In these setups, top_k is the variety of passages the retriever retains, and similarity_threshold is the minimal cosine rating a passage should attain to qualify. The code beneath declares all 4 as numbers to optimize:

# What groups usually write (and why it is the flawed exercise)
import optuna
def goal(trial):
    chunk_size    = trial.suggest_int("chunk_size", 100, 2000)
    chunk_overlap = trial.suggest_int("chunk_overlap", 0, 200)
    top_k         = trial.suggest_int("top_k", 1, 20)
    threshold     = trial.suggest_float("threshold", 0.5, 0.95)
    accuracy = run_rag_pipeline_and_score(
        chunk_size, chunk_overlap, top_k, threshold
    )
    return accuracy
examine = optuna.create_study(route="maximize")
examine.optimize(goal, n_trials=200)  # two weeks of compute later...

There’s a grain of fact right here. These variables do have an effect on retrieval high quality, and they’re price tuning. The difficulty begins with the phrase “hyperparameter,” which brings in a metaphor with hidden assumptions.

In machine studying, a hyperparameter controls how a mannequin learns: studying charge, regularization power, variety of layers. The mannequin itself is what modifications throughout coaching; the hyperparameter shapes that change. In RAG, there isn’t any studying. The chunk measurement doesn’t management how one thing learns. It controls how a perform splits textual content, the identical method each time, no matter what you’ve fed it earlier than.

What appears to be like like a hyperparameter is a configuration alternative, the type you’d make when configuring a search engine. The experience wanted to tune it effectively isn’t statistical optimization. It’s understanding the construction of your paperwork and the form of your questions. Chunk measurement of 512 tokens may fit fantastically on dense tutorial papers and disastrously on insurance coverage contracts the place a single clause spans 800 tokens and breaking it in half loses the conditional that offers the clause its that means. No grid search will let you know that. You’ll want to learn your paperwork.

This is the reason groups who grid-search chunk measurement typically discover a “greatest” worth that performs marginally higher on the check set and identically on manufacturing knowledge. The optimum on the check set was an artifact of the check set, not a real enchancment within the underlying system. They’ve optimized a quantity, not solved an issue.

Widespread pitfall: A workforce working Optuna over chunk_size, top_k, and similarity_threshold for 2 weeks, ending up at chunk_size=487 with no concept why. The sincere reply to “why 487?” is “as a result of Optuna stated so.” That reply doesn’t survive an actual manufacturing failure, and it doesn’t generalize when the doc distribution shifts. A bit measurement of 500 chosen as a result of that’s roughly the scale of a paragraph on this corpus is extra defensible than 487 chosen as a result of a sweep landed there.

The correct exercise isn’t tuning numbers. It’s deciding structurally easy methods to chunk. By part? By paragraph? By the desk of contents entries? By query kind, with completely different chunkers for brief lookups vs lengthy clauses? Answered by taking a look at paperwork and questions, not by optimization curves.

There’s a deeper purpose chunk measurement resists optimization: by development, no single chunk measurement can serve each query. Take two questions on the identical insurance coverage contract:

• “What’s the efficient date?” The reply is one line, someplace on web page one. It needs a bit sufficiently small to pin down a single line exactly.
• “What are the exclusions of the coverage?” The reply is perhaps one web page, or three pages, relying on how the insurer wrote it. It needs a bit massive sufficient to seize a complete part.

There is no such thing as a quantity that satisfies each. A bit measurement of 200 tokens chops the exclusions part into incoherent fragments. A bit measurement of 2000 tokens buries the efficient date in surrounding noise.

Looking for “the most effective chunk measurement” is subsequently not a tuning drawback. The framing itself is damaged: no single quantity can serve a distribution of questions whose solutions have completely different lengths.

You can, in precept, make chunk measurement reply to the query by coaching a small mannequin that predicts the best chunker from the query’s options: classify the intent, regress over the anticipated reply size, output a method. That may be machine studying utilized legitimately, on an issue the place one thing is being realized.

However you don’t have to. You may write the rule down. Have a look at a query and you may inform whether or not it asks for a date, a bit, or a comparability. So can a website professional. So can ten strains of Python with hand-written circumstances over key phrases. The deeper purpose RAG isn’t machine studying is that, for many of the choices contained in the system, you already know the reply, or somebody in your workforce does. Machine studying is the software for issues the place no one is aware of the reply prematurely.

The correct method is to cease on the lookout for one chunk measurement and begin routing completely different query varieties to completely different retrieval methods:

# What to do as an alternative: route by query kind
def chunk_for_question(query: str, line_df, toc_df):
    intent = classify_intent(query)
    if intent == "point_lookup":          # "what's the efficient date?"
        return chunk_by_line(line_df)
    elif intent == "section_retrieval":   # "what are the exclusions?"
        return chunk_by_toc_section(line_df, toc_df)
    elif intent == "comparability":          # "examine clauses A and B"
        return chunk_by_full_section(line_df, toc_df)

The 2 code blocks above are your complete argument of this part. The primary runs Optuna over 4 numbers for 2 weeks and produces a worth no one can defend. The second makes one structural choice per query kind and produces a system whose conduct anybody can clarify.

Later articles develop easy methods to classify intent (Article 6, on query understanding) and the way the completely different retrieval strategies and granularities are carried out (Article 7, on retrieval). The purpose right here is simply that the exercise isn’t tuning, it’s routing.

The following ML import is analysis methodology. The reasoning goes: RAG, like every ML system, wants a correct analysis dataset: questions paired with anticipated solutions, break up into practice and check units, scored with precision and recall. Frameworks like RAGAS have made this much more tempting, providing metrics for faithfulness, reply relevancy, and context recall that look satisfyingly ML-ish.

Analysis is beneficial. The difficulty isn’t whether or not to judge. It’s what the metrics imply. In machine studying, analysis tells you whether or not a mannequin has generalized from coaching knowledge to unseen examples. The practice/check break up exists since you wish to detect overfitting: a mannequin that memorized the coaching set moderately than studying a transferable sample.

In RAG, there may be nothing to generalize. Overfitting (when a mannequin memorizes coaching examples moderately than studying a sample that transfers to new knowledge) can not occur right here: the system doesn’t change between queries. The retriever computes the identical cosine distances each time. The generator follows the identical immediate template. There is no such thing as a mannequin adjusting to knowledge.

What analysis measures in RAG is three issues, all of that are protection and high quality questions, not statistical generalization:

• Does my corpus comprise the reply? If not, the system can’t discover it. It is a content material query, not a mannequin query.
• Does my retriever discover the best passage? If the reply is within the corpus however the retriever missed it, the system fails. It is a search query.
• Does my generator keep devoted to what was retrieved? If the best passage was retrieved however the mannequin paraphrased it incorrectly or hallucinated extras, the system fails. It is a technology self-discipline query.

Each factors to a particular repair. Mixing them up beneath an combination “accuracy” rating loses info. A 75% accuracy from “corpus is lacking 25% of the documented matters” calls for completely different motion than a 75% accuracy from “retriever misses the best passage 25% of the time.” The primary requires ingesting extra paperwork. The second requires fixing the retriever. An combination metric that treats them the identical hides the diagnostic.

This additionally explains why groups utilizing RAGAS-style frameworks typically report nice metrics on a held-out check set after which watch the system fail in manufacturing. The check set lined matters the place the corpus had solutions and the retriever occurred to seek out them. Manufacturing has questions whose solutions usually are not within the corpus in any respect, and the system both hallucinates or fails to say “not discovered.” The metric was excessive on the check set as a result of the check set was pleasant. The system isn’t damaged. The analysis was.

What it is advisable to consider, damaged down by query kind, takes about ten strains:

# Retrieval recall, per query, per intent
def evaluate_retrieval(reference_set, retrieve_fn):
    rows = []
    for ref in reference_set:
        retrieved_lines = retrieve_fn(ref.query)
        recall = len(set(retrieved_lines) & set(ref.expected_lines)) / len(ref.expected_lines)
        rows.append({
            "query": ref.query,
            "intent":   ref.intent,
            "recall":   recall,
            "hit":      recall > 0,
        })
    return pd.DataFrame(rows)
# All the time break down by query kind, by no means simply an combination
df.groupby("intent")["hit"].imply()
# point_lookup        0.92
# section_retrieval   0.41   <-- that is the true drawback
# comparability          0.55

A single combination accuracy of 63% would have hidden the disaster on section_retrieval. The per-intent breakdown reveals it immediately. Recall right here means: on questions the place the reply exists within the corpus, did the retriever discover the best passage? Grouping by intent (point_lookup, section_retrieval, …) exhibits which sort of query fails, and subsequently which half of the pipeline to repair.

RAG has two analysis surfaces with very completely different shapes.

The retrieval floor is a search drawback: did the best passage land in entrance of the mannequin? Measuring this implies checking, on a reference set of questions, whether or not the related strains or pages have been retrieved in any respect. The metric is recall on the stage you care about (recall at line, at web page, at part) and it’s particular to your corpus. No one else can run this analysis for you. Your corpus is exclusive. That is the place the majority of analysis effort belongs.

The technology floor is completely different. As soon as the best passage has been retrieved, the query turns into: did the mannequin produce a devoted reply, in the best format, with correct citations, and a clear “not discovered” when the passage didn’t comprise the reply? A few of this you do consider your self, however a big half is already evaluated by the LLM distributors. OpenAI, Anthropic, and Mistral spend huge sources testing whether or not their fashions observe JSON schemas, refuse to invent, and respect immediate directions. These are the scale on which they enhance their fashions. As a RAG builder, you’re not coaching the generator. You’re consuming it. If the mannequin fails badly at returning structured JSON or stays untrue to its inputs, you’ll discover inside an hour of integration. That’s not a metric to arrange; it’s a sanity verify that’s both apparent or high-quality.

What this implies in follow: most of your analysis time ought to go into retrieval (which is corpus-specific and solely you are able to do it), not into technology (which is generally the seller’s drawback, and which exhibits apparent failures quick). Groups that spend weeks constructing elaborate technology analysis suites are normally pushing aside the more durable retrieval work that will enhance the outcome.

Going additional: Evaluating Your System (later within the sequence) walks via easy methods to construct a reference set to your particular corpus, the 4 metrics that matter, and why per-question-type metrics are important whereas combination metrics are deceptive.

Machine studying has its personal toolkit for explainability. SHAP values to attribute predictions to options. LIME for native approximations of complicated fashions. Consideration visualization for transformers. When folks begin asking for RAG explainability (“why did the system give this reply?”) they naturally flip to those instruments. They wish to rating retrieval relevance, weight doc contributions, visualize which tokens influenced the output.

The irony is that RAG is extra explainable by design than most ML fashions. There’s no want for SHAP. There’s no opacity to crack open. The system retrieved these particular passages from these particular sources, and the reply was constructed on high of them. That is the reason. It’s documentary, not statistical.

This factors to a deeper asymmetry between machine studying and RAG. In machine studying, the human has instinct however can not quantify. Ask who survived the Titanic and folks say wealth, age, class: none flawed, none exact. The mannequin has no such doubt: match a choice tree and the basis break up is intercourse, the following lower is a precise age threshold no one would have guessed, then class. Each break up is a quantity instinct alone couldn’t have produced. The mannequin exists to place these numbers down.

For textual content knowledge, the route reverses. The person can learn the supply. A lawyer scanning a contract sees the circumstances, the exceptions, the dates. A compliance officer reads a coverage and is aware of whether or not a conduct breaches it. The textual content doesn’t conceal its that means, and the professional is already a fluent reader.

There are exceptions: sarcasm and irony are the traditional ones, the place trendy LLMs typically catch what a literal reader misses. However in enterprise contexts the person is the area professional.

The mannequin isn’t there to elucidate the textual content. It’s there to do the studying at corpus scale, and a quotation is sufficient to let the professional confirm any reply in seconds.

When a person asks “why this reply?”, the best response isn’t a heatmap of consideration weights or a characteristic attribution rating. It’s: “I checked out pages 12, 47, and 89 of this contract. Right here’s the precise textual content I used. The reply follows from that textual content.” If the person disagrees with the reply, they’ll learn the supply themselves and choose. They don’t want an explainability framework. They want a quotation.

The fifty-line pipeline from Article 1 already confirmed this. The immediate requested the mannequin to return the beginning and finish line numbers (with their pages) alongside the reply, in a structured JSON; the annotator then highlighted these actual strains on the PDF. No SHAP, no LIME, no consideration visualization, no specialised observability platform. The “rationalization” was a aspect product of how the immediate was written. The quotation is a part of the reply, not an evaluation layer added on high.

The hint is the reason. Studying it requires no interpretation, simply studying.

Importing ML explainability into RAG is fixing an issue that doesn’t exist. SHAP on a retrieval rating is utilizing a scalpel to open a mailbox. The retrieval rating is already a quantity you computed on inputs you’ll be able to learn. There’s nothing to attribute that you simply don’t already see.

The deeper failure of the ML-explainability framing is that it makes you give attention to the flawed factor. You begin attempting to elucidate why a specific passage scored increased than one other in vector area, a near-impossible query that doesn’t matter. What issues is whether or not the best passage was retrieved in any respect, and whether or not the reply faithfully displays it. These are questions you’ll be able to reply by studying the logs and the supply. No tooling wanted.

When you cease treating RAG as ML, two issues change. The day-to-day instruments, metrics and folks reorganize round search moderately than coaching. And a deeper query (the place the intelligence sits) strikes from the mannequin to the workforce. Each come from the identical framing.

Three concrete issues change.

The instruments change: You don’t want PyTorch, or a coaching cluster, or hyperparameter optimization frameworks for the system itself. You want a great parser, a versatile retriever, cautious immediate engineering, and structured logging of every part that occurs. The elements that are ML (the embedding mannequin, the LLM) you devour as companies. They’re commodity inputs, not stuff you construct or practice.

The metrics change: Mixture accuracy offers approach to per-failure-mode metrics: retrieval recall (did we discover the best passage?), reply faithfulness (did the mannequin stick with it?), extraction accuracy (when extracting structured knowledge, did the values match?), not-found charge (when the reply isn’t within the corpus, did we are saying so cleanly?). Every measures one thing particular, every maps to a particular a part of the pipeline you’ll be able to repair.

The folks change: A pure ML workforce attempting to ship a RAG system typically misses what makes it work, and what makes it fail. The talents that matter most are software program engineering (the system has many transferring components that have to compose cleanly), area experience (somebody has to know what a great reply to a website query even appears to be like like), and knowledge retrieval instinct (somebody has to suppose like a search engine designer, not a mannequin coach). ML experience is beneficial, however it’s not the dominant ability. A workforce of ML researchers and no area professional will produce a fantastically tuned system that misses the purpose. A workforce with one ML-aware engineer, two software program engineers, and one area professional will normally outperform it.

The shift in folks factors to a deeper query: the place does the intelligence of the system stay?

In an ML system the intelligence lives within the mannequin. The mannequin holds the patterns. The workforce feeds it coaching knowledge and tunes the loss perform. In a RAG system the intelligence lives within the workforce. The lawyer is aware of which clauses to have a look at first. The underwriter is aware of what “deductible” means, and which web page normally carries it. The compliance officer is aware of which regulation applies to which product. None of that lives contained in the embedding mannequin. None of it comes out of a hyperparameter sweep. It already lives within the heads of people that have learn these paperwork for years.

Watch an underwriter open a brand new coverage. She doesn’t learn it linearly. She jumps to the exclusions part first as a result of she’s learn 5 hundred of those and is aware of that’s the place the entice normally lives. She checks the schedule of advantages for the deductibles and ceilings. She checks the territory clause. Three minutes in, she has a clearer view of the contract than any embedding mannequin would produce on a thousand of these contracts. That behavior is what the system has to amplify.

The job of an enterprise RAG system is to amplify that experience at scale, not change it. What that appears like will depend on the brick.

Parsing comes first. If the parser turns a contract’s PDF into scrambled textual content, no downstream cleverness recovers it. If the doc has a working desk of contents, the parser has to extract it cleanly, as a result of the TOC is what the professional depends on to navigate. When a doc has no TOC in any respect (scanned faxes, slide decks exported to PDF, outdated typewritten insurance policies), reconstructing one turns into a job in itself, typically extra helpful than any retrieval tweak.

Query understanding carries the workforce’s vocabulary throughout the hole between how a person phrases a query and the way the doc writes the reply. The pilot person varieties kettle, the contract says small electrical equipment. The compliance officer varieties knowledge breach, the coverage says unauthorized disclosure of non-public info. The professional is aware of the mapping. The query parser turns that mapping right into a lookup desk: translations throughout languages, spelling variants, plural types, inside acronyms. None of it’s realized from knowledge, it’s dictated by the professional and written down.

Retrieval amplifies what the professional already does by hand. The professional searches key phrases; that half is already straightforward. What the professional can not do at scale is run regex patterns over hundreds of pages, verify whether or not two phrases co-occur inside the identical paragraph, or mix boolean circumstances throughout the entire corpus. The retriever does that work quick, then fingers candidates again so the professional can confirm.

Technology does the 2 issues the professional would in any other case do by hand: cite the precise passage that helps the reply, and format the uncooked worth into one thing usable. The string 3455434 on the web page turns into €3,455,434 within the reply. 20260516 turns into Could 16, 2026. thirty days from the date of the loss stays verbatim, with a quotation again to the clause so the professional can confirm in a single click on.

Articles 5, 6, 7, and eight develop every brick in flip: the parser that extracts TOC construction, the professional dictionary that maps vocabulary, the TOC-aware retriever, the typed-answer generator. Identical precept each time: choose up a chunk of human experience and transfer the repetitive half to the machine.

That is additionally why the sequence is cautious with autonomous brokers. It prefers key phrase retrieval to embedding similarity by default. It treats reranker tuning as a final resort. Every of these defaults assumes there isn’t any professional to seek the advice of. In enterprise contexts the professional is at all times there. The system ought to take heed to them.

Should you work in a setting with no professional, with unbounded questions, with very completely different paperwork, this sequence won’t be your greatest information. Normal-purpose retrieval and autonomous brokers are a greater match there.

A helpful approach to image RAG is as a search engine, plus an LLM that writes the reply. Two components, every with a transparent job, every with its personal method of breaking.

The search engine retrieves passages from paperwork. Given a query, return the strains, paragraphs, or sections most definitely to comprise the reply. It is a pure search drawback: selectivity, recall, rating. Many years of knowledge retrieval concept apply. The truth that a part of it makes use of neural embeddings doesn’t change its nature; embedding similarity is only one rating sign amongst a number of.

The LLM takes a passage and a query and produces a natural-language reply with a quotation. The LLM doesn’t discover the reply. The search engine already did that. The LLM writes the reply from a passage that’s been positioned in entrance of it. It’s nearer to a translator or a scribe than to an oracle.

Mapping again to the 4 bricks from Article 1: parsing, query understanding, and retrieval collectively make up the search engine; technology is the LLM. The brick view is the operational one (one field of code per brick); the two-part view is the psychological mannequin you carry in your head when one thing goes flawed.

The 2 components fail in several methods, and the analysis begins on the seam between them. Pull the hint from a failing question: have been the retrieved passages in entrance of the mannequin, and did they comprise the reply?

If the reply wasn’t within the retrieved passages, the search engine is the perpetrator, and the repair is upstream. Was the best web page corrupted by the parser (OCR errors, multi-word phrases break up throughout strains, two-column interleaving)? Did the query parser miss a synonym the professional vocabulary ought to have expanded? Did the retrieval mechanism rank the best web page out of top_k, or break on punctuation that wanted a regex? Or is the related doc simply not within the corpus? 4 very completely different fixes, all upstream. “Tune the retriever” is meaningless till you’ve localized which one. The identical 4 bricks that amplify the professional when working (part 3.3) break in their very own methods right here, every with its personal deep-dive article (Articles 5, 6, 7).

If the reply was within the retrieved passages however the response is flawed, the LLM is the perpetrator, and the repair is downstream. Widespread patterns: the mannequin paraphrased and misplaced a conditional, returned the uncooked 3455434 as a result of the schema left the reply free-form, cited the flawed line numbers, invented a worth not within the passage, or produced a solution when it ought to have stated “not discovered”. 5 technology bugs, 5 completely different fixes, all within the immediate, schema, or post-validation layer (Article 8). None of them get higher by tuning the retriever.

A workforce at a mid-size enterprise is given six months to ship a RAG system over a number of thousand inside paperwork. They begin by constructing an analysis dataset of 500 questions, splitting it 70/30 into practice and check. They arrange Optuna to comb chunk measurement, overlap, top-k, and similarity threshold. The primary sweep takes per week of compute, comes again with a “greatest” configuration, and the workforce ships it for inside testing.

The pilot customers complain instantly. The system solutions fluently however is flawed half the time on questions that the evaluators clearly know: questions on particular clauses, particular dates, particular numerical limits. The workforce’s response is to broaden the analysis dataset, run one other sweep, fine-tune the embedding mannequin on artificial question-document pairs, and add a reranker. Three extra months go by. Manufacturing accuracy doesn’t transfer.

What was flawed: the parser was treating scanned pages with degraded OCR layers as in the event that they have been native textual content. About 30% of the corpus was successfully unreadable, however the workforce’s analysis set occurred to be drawn from the readable 70%. No quantity of chunk measurement optimization, embedding fine-tuning, or reranker integration may repair it: a 3rd of the paperwork have been producing rubbish. A two-day funding in checking every web page (the work of Article 5, on parsing) would have caught this on day one.

The workforce had spent six months in ML mode (sweeping hyperparameters, rising analysis units, fine-tuning fashions) when the repair was a parser change.

This story is composite, however each factor of it has occurred in actual initiatives. The sample is constant: ML reflexes drive the workforce towards optimization actions that really feel productive, whereas the structural issues sit untouched within the parser, the corpus, or the not-found logic. The primary intuition on a struggling RAG system shouldn’t be “let’s tune”. It ought to be “let’s hint what occurs to a failing question, finish to finish, and discover the damaged hyperlink.”

RAG appears to be like like machine studying. The resemblance is shallow. The reply exists within the doc or it doesn’t. There is no such thing as a statistical generalisation, no studying curve, no practice/check break up that maps to actual failures. The correct framing is search engine meeting: a search engine plus an LLM, two components you’ll be able to repair independently, with per-failure-mode metrics changing combination accuracy.

The price of holding on to the ML framing just isn’t mental. It’s six months of cautious work on the flawed drawback. Article 4 turns the best framing right into a working diagnostic: RAG issues sit on a grid of doc complexity by query management, and every cell requires a distinct stack.