RAG Workflow and Loop Engineering: The Dispatcher That Decides When to Loop and When to Cease

A workflow like this wants precisely one new house within the codebase: the composition layer. It holds the items that exist solely as a result of bricks are being mixed: the dispatcher that picks which patterns fireplace, the suggestions equipment that reads the typed flags, and one movement per intent. Nothing else strikes. The 4 bricks hold residing in their very own modules, and the patterns of Half III keep the place they belong as sub-functions of their brick: TOC retrieval within the retrieval module, escalation in parsing, two-hop decision throughout retrieval and era. The 2 layers speak by way of typed objects (a ParsedQuestion, a DocumentProfile, an AnswerWithEvidence), by no means by way of one another’s internals.

One method to lay that out on disk, the one this sequence makes use of: a pipeline/ folder sitting subsequent to the brick modules:

Contained in the composition layer, code is organised by intent (one subpackage per intent, one file per format inside), with the cross-intent infrastructure flat on the root; Article 18 (code structure) develops the file structure intimately.

This break up makes the system maintainable. Once you enhance TOC retrieval, you alter the retrieval module solely. Once you repair a bug in two-hop decision, the orchestrator stays untouched. The contract between orchestrator and bricks is the interface (a ParsedQuestion, a DocumentProfile, an Activations dict, an AnswerWithEvidence), not the implementation.

A senior engineer can hint any request from enter to highlighted reply in a couple of minutes: pdf_qa_loop (pipeline/qa/pdf_flow.py) reads the query, calls decide_pipeline_patterns for activations, runs the four-brick cross (the identical parse and retrieval entry factors as Article 9’s pdf_qa) with these activations, reads the era suggestions fields, and both returns the reply or queues a loop. No hidden state, no agent loop.

The file structure has a runtime twin: how the handlers nest when a request really runs. Every layer calls the one inside it and provides precisely one concern, and the management loop stays in code at each degree.

The orchestrator does three issues, so as:

1. It reads the parsed query (ParsedQuestion from Article 6) and decides which patterns to activate. Some questions want all of them. Most don’t.

2. It runs the lively patterns, passing the fitting inputs to every. The order issues: TOC retrieval earlier than key phrase retrieval (so structural anchors come first), major retrieval earlier than two-hop (so the two-hop step has one thing to observe from), and so forth.

3. It manages suggestions loops. When the era step indicators an incomplete reply, the orchestrator decides whether or not to iterate and what number of instances. A number of loops can fireplace on one query (parsing escalation, vocabulary enlargement, reference enlargement, itemizing iteration). The orchestrator picks the order throughout iterations (parsing first, vocabulary subsequent, references subsequent, itemizing final) and shares one finances throughout all of them. Two to a few iterations cowl the overwhelming majority of instances; the composite’s laborious cap is 4, as a result of a number of loops can fireplace on one query and every wants its personal cross. Order issues as a result of re-retrieving on unhealthy parsing wastes the iteration; higher key phrases enhance each later step.

Concretely:

# pdf_qa_loop, condensed
# decide_pipeline_patterns and iterate_with_bound come from the companion pocket book.

def pdf_qa_loop(pdf_path, query, *, registry=None, max_iterations=4):
    registry = registry or PatternRegistry.default()
    # 1. Query understanding
    parsed = registry.parse_question(query)
    # 2. Low-cost layer-1 parse of the doc
    line_df, page_df, toc_df = registry.parse_layer1(pdf_path)
    # 3. Doc profile
    doc_profile = registry.detect_document_type(pdf_path)
    # 4. Dispatcher : which patterns fireplace ?
    activations = decide_pipeline_patterns(parsed, doc_profile)
    # 5-7. Retrieval + era inside a bounded suggestions loop
    state = _State(parsed=parsed, line_df=line_df,
                   page_df=page_df, toc_df=toc_df)
    consequence = iterate_with_bound(
        initial_state=state,
        run_pass=lambda s: _one_pass(s, registry, activations, pdf_path),
        is_satisfactory=lambda a: not _needs_iteration(a),
        alter=lambda s, a: _adjust_state(s, a, registry,
                                          activations, pdf_path),
        document=_build_record,
        max_iterations=max_iterations,
    )
    return CompositeOutput(reply=consequence.end result, activations=activations,
                           historical past=consequence.historical past,
                           exhausted=consequence.exhausted)

That’s the entire orchestrator. The pdf_qa_loop physique is a web page of glue code sitting on high of the per-pattern modules. The third bullet (suggestions loops) has its personal devoted equipment; part 3 walks by way of it. The primary two bullets (sample choice, run order) are deterministic as soon as decide_pipeline_patterns returns its column of ON/OFF flags; part 4 walks by way of the dispatcher itself.

The composite pipeline’s distinguishing characteristic, in comparison with a naive RAG, is that the reply is handled as a provisional output that the system itself critiques. The loop equipment sits between era and the ultimate return.

A naive RAG pipeline runs the 4 bricks in sequence and returns no matter comes out: parse the doc, parse the query, retrieve, generate.

If retrieval missed the fitting passage, the reply is incorrect. If parsing was inadequate, the LLM hallucinates across the hole. The consumer by no means is aware of.

The composite pipeline provides a critique step on the finish of the row, with two suggestions rails that loop again every time the LLM indicators an issue: again into retrieval when the reply is incomplete (complete_answer_found=false, increase the key phrases and re-retrieve), and again into parsing when the context got here out unstructured (context_structured=false, re-parse the flagged pages):

The critique step reads the structured suggestions fields the LLM produced (Article 8): is the context full? did the LLM uncover new key phrases throughout studying? was the parsing ample? If something is off, the pipeline takes focused motion and re-generates.

These rails are the pipeline’s large loops: those that cross bricks, triggered by era studying its personal enter (adaptive parsing in Article 10, reference decision in Article 11, scope suggestions right here). Every brick additionally runs its personal small bounded loops inside (the picture cascade in parsing, the TOC descent in retrieval, the schema retry in era); these by no means attain the orchestrator. The dispatcher owns the massive ones solely.

That is what most distributors now market as agentic RAG. The time period covers three fairly various things; part 6 kinds them out.

Every loop has three dimensions: the sign (what the LLM or a programmatic examine reviews), the set off (which state of the output begins a retry), and the motion (what the pipeline modifications for the following cross). Along with the bounds of part 3.3, these are the three management surfaces each loop within the sequence wears: sign plus set off resolve when a brand new cross fires, the motion is the restoration (the one factor that modifications earlier than the retry), and the bounds are the termination.

Indicators come from three sources:

• LLM self-assessment: Fields like complete_answer_found, confidence, caveats. Mushy sign, broadest protection.
• Programmatic checks: Deterministic code validates citations, format constraints, cardinality. Stronger sign as a result of the examine itself doesn’t rely on the LLM’s judgment, regardless that it runs on the LLM’s output.
• Exterior validation: A separate mannequin, a downstream device, or a human. Strongest, most costly. Reserved for high-stakes outputs.

Triggers map to particular actions (every was launched as soon as in Half II or III; right here is the consolidated catalogue):

• Incomplete reply (complete_answer_found=False from Article 9’s AnswerWithEvidence, or is_likely_complete=False from Article 12’s ListAnswer) → increase retrieval scope or activate a brand new sample (itemizing, two-hop).
• Inadequate context (context_structured=False, Article 10) → set off adaptive parsing on the failing pages.
• Found vocabulary (llm_discovered_keywords populated, Article 8) → re-retrieve with the expanded vocabulary.
• Pending references (pending_references non-empty, Article 11) → run two-hop retrieval on every reference.
• Cardinality mismatch (Article 12) → iterate with expanded key phrases or a broader part restriction till the rely matches or a certain is hit.
• Failed validation (any programmatic examine failed) → re-retrieve or ask the LLM to right.

Every set off maps to a named, focused motion, which retains the loop debuggable.

The primary danger of iterative pipelines is unbounded iteration. Three controls.

Most iterations: A ceiling on pdf_qa_loop’s max_iterations parameter. Two or three iterations cowl the overwhelming majority of instances. Past that, returns diminish sharply.

def iterate_with_bound(
    *,
    initial_state: TState,
    run_pass: Callable[[TState], TResult],
    is_satisfactory: Callable[[TResult], bool],
    alter: Callable[[TState, TResult], TState],
    document: Callable[[int, TResult, TState, TResult | None], IterationRecord],
    max_iterations: int = 3,
) -> IterationOutcome[TState, TResult]:
    """Run a bounded pipeline-with-feedback loop."""
    state = initial_state
    historical past: record[IterationRecord] = []
    end result = run_pass(state)
    previous_result: TResult | None = None
    if is_satisfactory(end result):
        historical past.append(document(0, end result, state, previous_result))
        return IterationOutcome(end result=end result, historical past=historical past, exhausted=False)
    for i in vary(1, max_iterations):
        next_state = alter(state, end result)
        historical past.append(document(i, end result, next_state, previous_result))
        previous_result = end result
        state = next_state
        end result = run_pass(state)
        if is_satisfactory(end result):
            return IterationOutcome(end result=end result, historical past=historical past, exhausted=False)
    return IterationOutcome(end result=end result, historical past=historical past, exhausted=True)

A loop that hits max_iterations doesn’t crash. It returns one of the best output produced up to now, flagged so the consumer and the audit path know iteration was exhausted.

Termination situations: Past max iterations, the loop stops when:

• No new sign: The identical set off fires twice with the identical recommended motion.
• No new candidates: Retrieval returned the identical set of passages as final iteration.
• Lowering confidence: The loop is making issues worse, not higher. Cease iterating and return the highest-confidence reply seen up to now.

def should_continue(historical past: record, present: _ResultProto,
                    confidence_drop_threshold: float = 0.1) -> bool:
    """Determine whether or not the iteration loop ought to run one other cross.

    Three causes to cease, so as of precedence:
      1. Candidates are secure (similar set as final cross): nothing new shall be discovered.
      2. Advised key phrases are secure: the LLM is repeating itself.
      3. Confidence is lowering previous the brink: the loop is making issues worse.
    """
    if historical past:
        prev = historical past[-1]
        if getattr(prev, "candidates", None) == present.candidates:
            return False
        if getattr(prev, "suggested_keywords", None) == present.suggested_keywords:
            return False
        prev_conf = getattr(prev, "confidence", None)
        if prev_conf will not be None and present.confidence < prev_conf - confidence_drop_threshold:
            return False
    return present.needs_iteration()

With out these, even with a max-iterations certain, the pipeline burns three full iterations on instances the place it had clearly stopped progressing after the primary.

Drift detection: A subtler failure: the loop iterates efficiently (every cross finds new candidates) however the reply drifts away from the unique query. Key phrase enlargement picked up a tangent. The consumer requested about “premium”, the LLM noticed “insurance coverage”, expanded to “coverage”, then to “protection”, then to “reinsurance”, and the reply finally ends up about reinsurance economics.

The repair is to maintain the unique query in scope at each iteration. Expanded key phrases add to the retrieval, they don’t substitute the unique anchors. Technology at all times sees the unique query.

def expand_query_safely(parsed, new_keywords: record[str],
                        max_keywords: int = 15):
    strive:
        expanded = parsed.model_copy(deep=True)
    besides AttributeError:
        import copy as _copy
        expanded = _copy.deepcopy(parsed)

    originals = record(expanded.retrieval.anchor_keywords or [])
    additions = [k for k in new_keywords if k not in originals]
    mixed = originals + additions
    if len(mixed) > max_keywords:
        kept_originals = originals[:max_keywords]
        room = max_keywords - len(kept_originals)
        mixed = kept_originals + additions[-room:] if room > 0 else kept_originals
    expanded.retrieval.anchor_keywords = mixed
    return expanded

A drift examine at every iteration: if the brand new candidates have low overlap with the authentic anchor key phrases, that’s drift, and the loop ought to cease.

5 anti-patterns to maintain out of the orchestrator:

• Iterating on confidence alone: Confidence is noisy. An LLM will be extremely assured on a incorrect reply or low confidence on an accurate one. Use confidence as one sign amongst a number of, by no means alone.
• Unbounded key phrase enlargement: Cap the key phrase set dimension (10 to fifteen) and prune the least informative when the cap is hit. In any other case after three iterations the retrieval question is fifty phrases lengthy and signal-to-noise collapses.
• Re-parsing greater than crucial: When context_structured=False is signaled, re-parse solely the pages the LLM flagged, not the entire doc.
• Hiding iteration from the audit path. Each iteration should produce an IterationRecord. Compliance contexts require this; debugging contexts profit from it.
• Deciding to iterate purely with the LLM. The LLM can contribute sign to the iteration resolution however the resolution to iterate or cease have to be in code, with express guidelines. should_continue (above) is the place these guidelines stay. The dispatcher decides solely which patterns to fireplace upfront, not whether or not to loop.

{
  "iteration_number": 2,
  "set off": "vocabulary_gap",
  "action_taken": "re-retrieve with `extra`",
  "result_summary": "web page 7 joined the candidate set; confidence rose by 0.22",
  "confidence_before": 0.62,
  "confidence_after": 0.84
}

The complete iteration historical past travels with the ultimate reply. For audit and compliance, that is the distinction between “the system gave this reply” and “the system gave this reply by way of these particular steps, every justified by an express set off.” The UI presents a one-line narration per iteration so the consumer sees the system making knowledgeable decisions reasonably than operating blind retries.

The dispatcher (resolve.py) is the place a lot of the express selections occur. It takes the parsed query and the doc profile, and returns a dictionary of sample activations.

def decide_pipeline_patterns(
    parsed: ParsedQuestion,
    doc_profile: DocumentProfile,
) -> Activations:
    activations = dict(DEFAULT_ACTIVATIONS)

    # TOC retrieval: allow every time the doc has a usable TOC.
    if doc_profile.has_usable_toc:
        activations["toc_retrieval"] = True

    # Dense retrieval (embeddings): solely as a fallback for questions whose
    # vocabulary might not match the doc's wording.
    if parsed.intent in ("open_scoped", "open_corpus_wide"):
        activations["dense_retrieval"] = True

    # Two-hop references: allow when the query carries a structural
    # trace that factors outdoors its major passage.
    if parsed.retrieval.section_hint or parsed.retrieval.layout_hint:
        activations["two_hop_references"] = True
    if parsed.intent in ("section_retrieval", "open_scoped"):
        activations["two_hop_references"] = True

    # Itemizing aggregation: allow when the query intent is itemizing.
    if parsed.intent == "itemizing":
        activations["listing_aggregation"] = True

    return activations

This operate encodes the crew’s understanding of which questions want what. Because the system runs in manufacturing, it evolves: new query sorts, new heuristics, edge instances noticed in analysis feed again into the dispatcher.

The dispatcher can also be the file the crew ought to iterate on with care. A foul activation rule can both over-engineer easy questions (operating itemizing aggregation when not wanted) or under-engineer advanced ones (skipping two-hop on questions that want it). Every rule ought to have a check case hooked up.

The orchestrator’s two inputs are a ParsedQuestion (from Article 6) and a DocumentProfile (an affordable probe on the PDF). Each are constructed beneath from actual code. detect_document_type opens the Transformer PDF; the ParsedQuestion is what the LLM-backed parse_question would produce for the query above.

{
  "original_question": "What regularization strategies does the Transformer paper use, and the place do they present the impression on BLEU?",
  "key phrases": ["regularization", "dropout", "label smoothing", "BLEU"],
  "intent": "itemizing",
  "retrieval": {
    "main_query": "regularization strategies and their impression on BLEU",
    "rewrites": ["Residual Dropout", "Attention Dropout", "Label Smoothing", "Pdrop", "label smoothing epsilon", "Table 3"],
    "anchor_keywords": ["regularization", "dropout", "label smoothing", "Table 3"],
    "section_hint": "5.4",
    "layout_hint": "desk"
  }
}

The three fields the dispatcher reads: intent = "itemizing", section_hint = "5.4", layout_hint = "desk". The primary prompts itemizing aggregation; the opposite two activate two-hop references.

{
  "total_pages": 15,
  "has_usable_toc": true,
  "is_likely_scanned": false,
  "suggested_strategy": "native_text_only"
}

The profile is affordable: one fitz.open name, no physique parsing. Web page rely, TOC presence (has_usable_toc = True as a result of the Transformer paper carries 22 bookmarks), and a scan flag (False right here, because the PDF is native textual content). The dispatcher reads has_usable_toc to allow TOC retrieval.

decide_pipeline_patterns(parsed_q, doc_profile) reads the 2 objects above and returns an activation column. The output beneath is the true end result from operating the dispatcher on this query and this doc:

{
  "toc_retrieval": true,
  "keyword_retrieval": true,
  "dense_retrieval": false,
  "two_hop_references": true,
  "listing_aggregation": true,
  "adaptive_parsing": false,
  "iterative_feedback": true
}

4 out of seven patterns activated (plus iterative suggestions because the always-on security internet). The 2-hop sample makes this query totally different from a plain itemizing.

pdf_qa_loop runs its cross (the identical parse and retrieval entry factors as Article 9’s pdf_qa) with the activation flags above. Inside that cross:

• TOC retrieval (Article 9) jumps to Part 5.4 “Regularization” (web page 7). The key phrase retrieval, anchored on regularization, dropout, label smoothing, reinforces pages 7-8 (the place the daring headers Residual Dropout and Label Smoothing sit). RRF fusion ranks pages 7-8 first.
• Itemizing aggregation (Article 12) restricts to Part 5.4 and picks up three strategies: sub-layer dropout (Pdrop = 0.1, on the output of every sub-layer earlier than add+layernorm, web page 8), embedding dropout (Pdrop = 0.1, on the sum of token and positional embeddings, web page 8), and label smoothing (εls = 0.1, web page 8). The primary two share the Residual Dropout paragraph within the paper; label smoothing is its personal paragraph. The cardinality cue “We make use of three varieties of regularization throughout coaching” on the backside of web page 7 confirms the rely.
• First era cross produces a solution with the three strategies however no validation numbers. The LLM units complete_answer_found=False and pending_references=["Table 3"] as a result of the prose says outcomes are reported in Desk 3 however Desk 3 wasn’t within the candidate set.

The pipeline doesn’t return a solution but. The pending_references set off fires.

The orchestrator’s suggestions equipment sees pending_references = ["Table 3"] and runs the two-hop sample (Article 11). Web page 9 carries Desk 3: Variations on the Transformer structure. The related rows for this query are:

• Row (D): varies Pdrop at 0.0 / 0.1 / 0.2.
• Row (E): varies εls at 0.0 / 0.1 / 0.2, plus a positional-embedding variant.

Rows (A), (B), (C) range consideration heads, key dimension, and mannequin dimension and will not be related to the regularization query. The 2-hop retriever pulls web page 9 and joins it to the candidate set, then pdf_qa re-runs era with Part 5.4 plus Desk 3 within the context:

The Transformer paper makes use of three regularization strategies throughout coaching: 1. Residual Dropout -- Pdrop = 0.1, web page 8 (Part 5.4, daring header) Validated in Desk 3 row (D), web page 9: various Pdrop modifications BLEU on newstest2013 (4.92 PPL / 25.8 BLEU on the base 0.1 worth). 2. Consideration Dropout -- Pdrop = 0.1, web page 8 (similar paragraph as Residual Dropout) Identical Desk 3 row (D) covers it: Pdrop is utilized uniformly. 3. Label Smoothing -- epsilon_ls = 0.1, web page 8 (Part 5.4, daring header) Validated in Desk 3 row (E), web page 9: various epsilon_ls modifications BLEU. The textual content notes this "hurts perplexity however improves accuracy and BLEU". Cardinality: Part 5.4 opens with "We make use of three varieties of regularization throughout coaching" (web page 7). All three are current on this record.

complete_answer_found=True, pending_references=[]. The orchestrator stops. The reply is structured (3 enumerated strategies), exhaustive (cardinality cue confirmed), and full (every method has each its definition web page and the Desk 3 row that varies it).

Indicative price form for a 15-page native PDF on a typical OpenAI-class mannequin. The numbers are tough; what issues is the form: the LLM calls dominate, all the pieces else is sub-second.

• Low-cost parsing (PyMuPDF on 15 pages): properly beneath one second.
• Doc profile (one fitz.open + TOC learn): ~50 ms.
• Mixed retrieval (TOC + key phrase + RRF fusion): just a few hundred ms.
• Itemizing aggregation + cardinality examine: just a few hundred ms.
• Two-hop reference decision (one regex sweep + a re-retrieve on web page 9): beneath one second.
• Two era calls (gpt-4-class): ~2 to 4 seconds every.
• Whole: usually 5 to 9 seconds for a query that prompts 4 patterns and loops as soon as on a reference. The 2 era calls drive nearly all of it.

With out the dispatcher, you’ll both pay for each sample on each name, or hand-code the routing for each query form. Dense retrieval and adaptive parsing stayed OFF right here as a result of the profile mentioned they weren’t wanted; that call sits in resolve.py, not within the query itself.

Part 3 famous that the feedback-loop form is what most distributors name agentic RAG. The time period is used three alternative ways, and selecting between them shapes the entire structure.

Utilization 1: agentic as advertising and marketing: Any pipeline extra refined than embed → retrieve → generate will get known as agentic. The label is empty. We are able to ignore this utilization.

Utilization 2: feedback-driven management: The LLM produces sign (complete_answer_found, llm_discovered_keywords, context_structured), and deterministic Python code reacts to that sign. That is what we’ve constructed. The LLM is within the system however not accountable for the system. Management stays within the dispatcher.

Utilization 3: autonomous brokers: The LLM has a set of instruments and decides at every step which one to name subsequent. The management loop is within the LLM itself. The Python code executes regardless of the LLM requests.

The excellence between Utilization 2 and Utilization 3 is the architectural resolution that issues most for enterprise RAG.

In Utilization 3, the orchestrator above would get replaced by one thing like:

# Autonomous agent - what we're NOT constructing
instruments = [retrieve_by_toc, retrieve_by_keywords, retrieve_dense,
         re_parse_with_camelot, follow_reference, ask_clarification]

state = {"query": query, "historical past": []}
whereas not is_satisfied(state):
    next_action = llm.choose_next_tool(state, instruments)
    end result = execute(next_action)
    state["history"].append((next_action, end result))

The LLM decides which device to name, with which arguments, during which order. The pipeline is generated at runtime by the LLM, not authored by the engineer.

That is interesting in principle: the LLM can mix instruments in methods the engineer didn’t anticipate. In follow, for enterprise RAG, three issues go incorrect.

Reproducibility breaks: The identical query on the identical doc can take totally different paths on totally different runs. For audit, compliance, authorized evaluate, that is disqualifying. “Why did the system retrieve this passage?” has no single reply when the LLM selected the trail freely.

Prices explode: Every device resolution is an LLM name. 5 to 10 device calls per query. At lifelike scale, the fee is one to 2 orders of magnitude increased than the dispatcher sample. For a system answering hundreds of questions per day, that is the distinction between reasonably priced and not possible.

Debugging turns into guesswork: When an autonomous agent produces a nasty reply, you must learn its reasoning hint to know what went incorrect. The hint is lengthy, branchy, and never at all times coherent. In comparison with the dispatcher sample, the place you’ll be able to pinpoint precisely which activation rule fired or didn’t, autonomy is a step backward in maintainability.

The precious a part of the agentic concept isn’t autonomy. It’s that the pipeline thinks in regards to the query earlier than looking, and reacts to its personal outputs. Each of these are current in our composite pipeline:

• The pipeline thinks in regards to the query (Article 6, query understanding).
• The pipeline reacts to its personal outputs by way of suggestions fields (Article 8, era suggestions).
• The pipeline iterates when wanted (part 3 above, bounded by the orchestrator).

What we don’t hold is the autonomy of the LLM in selecting device sequences. That call belongs to the dispatcher, in code, the place it’s testable, auditable, and reproducible.

A extra trustworthy identify is structured RAG or dispatched RAG. “Agentic” survives as a result of the sphere has settled on it, however the structure stays in code, not within the LLM.

Autonomous brokers have their place. They’re the fitting sample when:

• The set of instruments is open and modifications steadily. A analysis assistant that should mix retrieval, net search, code execution, calculator, and many others., throughout very totally different query sorts.
• The query house is simply too diverse to dispatch deterministically. “Assist me plan this journey” legitimately wants totally different mixtures of instruments every time.
• Reproducibility isn’t a tough requirement. Client functions, exploratory tooling, inside R&D.

For enterprise RAG on paperwork, which is what this sequence is about, the construction of the issue is well-understood. We all know what a doc appears to be like like, what sorts of questions get requested, what sorts of patterns assist. The dispatcher sample captures that data. Autonomy provides price and unpredictability with out including functionality.

Dispatched RAG has a property that autonomous RAG doesn’t: the crew’s understanding of the issue is encoded within the dispatcher. When a brand new engineer joins, they learn resolve.py and learn the way the crew thinks about query sorts and sample activations. When a query fails in manufacturing, the crew provides a check case and updates the dispatcher. When a brand new sample is added (say, table-comparison retrieval for cross-document questions), it will get a brand new activation rule.

An unaudited autonomous agent has no such artifact: its conduct emerges from the LLM’s reasoning at runtime, which doesn’t accumulate, doesn’t get reviewed, and doesn’t switch between crew members. That is a part of why the dispatcher sample has held up in manufacturing: the dispatcher is one thing a crew can personal, evaluate, and hand off. The layer that sits on high, the place the agent picks amongst patterns like those dispatched right here, is follow-up work.

The architectural dedication that holds the 4 bricks collectively is one sentence: selections stay in resolve.py, not in an LLM immediate at runtime. The orchestrator runs the bricks so as, the feedback-loop equipment turns era’s structured output right into a bounded retry resolution, and the dispatcher picks the patterns from the parsed query and the doc profile.

The remaining articles tackle what surrounds the pipeline reasonably than what’s inside: scaling from one doc to a corpus (Half IV), and evaluating, operating, storing, and securing the system in manufacturing (Half V).