Loop Engineering with Adaptive PDF Parsing: Begin Low cost, Pay for a Heavier Parser Solely When the Web page Wants It

Adaptive parsing runs in two phases. Section one is initialisation: choose a baseline parser per doc, primarily based on what the doc is (native PDF, scan, Phrase export). In a platform context this fires at ingestion: each new doc will get its baseline parse on arrival, so the system all the time has a primary textual layer to work with, even for scanned PDFs the place the baseline must be an OCR engine as a result of PyMuPDF would return empty. Section two is the cascade: run low cost deterministic checks on the baseline’s output and escalate the pages that fail to a deeper parser. Most pages keep on the baseline; the few that fail a verify get the richer therapy they really want.

The pipeline all the time begins low cost on part one. PyMuPDF on native PDFs, a free OCR engine on scans. Each produce the identical line_df form, so the remainder of the pipeline (query parsing, retrieval, era) doesn’t know which parser ran. That works more often than not. It fails on a small however predictable set of content material shapes, and part two has a number of locations to catch the failure, ordered from low cost deterministic checks on the parser’s outputs to a final-line-of-defence LLM name at era time.

Low cost parsing is quick and free, and it leaves a recognisable set of content material invisible or scrambled. 5 shapes come up usually:

• Flattened tables: The most typical failure on native PDFs. PyMuPDF returns every cell as a separate line, in geometric order, with column construction erased. “Self-Consideration” and “O(n² · d)” sit on traces 9 and 10 of the web page with nothing connecting them: the LLM can’t pair the row label with its worth. Detected at parsing-time by the flat-table fingerprint (Part 3.2.2); walked end-to-end within the second half.
• Figures and diagrams: PyMuPDF extracts the web page textual content round a picture however the picture itself is opaque to it. “Determine 1 reveals the encoder-decoder structure” survives; the structure diagram doesn’t. Detected at parsing-time by the opaque-figure verify (Part 3.2.3); walked end-to-end within the second half.
• Multi-column layouts: A two-column web page the place the parser walks columns geometrically as a substitute of logically produces interleaved sentences from each columns. The textual content is technically there; the studying order is damaged.
• Degraded OCR: Scanned pages with low decision, skew, or compression artefacts produce character-level errors: l/I/1 confusion, fragmented phrases, lacking punctuation. The LLM can generally get well the which means, generally hallucinates across the noise.
• Embedded objects: Equations rendered as vector paths, signatures as photographs, checkboxes, watermarks: objects the parser might return as placeholders, garbled textual content, or nothing in any respect.

This half picks two of those (flattened tables, figures) for the labored walkthroughs the second half develops, as a result of they’ve the cleanest reproductions on a public doc. The mechanism is identical for the opposite three. The parsing-time deterministic checks (doc parsing brick, part 3) flag the failure form, and the pipeline routes the web page to the proper deeper parser. The LLM sign at era time (era brick, within the second half) is the security internet for the circumstances the deterministic checks didn’t catch upstream.

Another reason to parse lazily, in numbers. An enterprise corpus averages 30 to 100 pages per doc, 1 to three pages related to any given query, and fewer than 10 questions per doc over its lifetime. Multiply these out and roughly 90% of any web page parsed at ingestion isn’t consumed by any reply. Deep-parsing each web page upfront pays the invoice for pages nobody will ever learn.

Right here is the entire article in a single image. The remainder of the textual content walks each bit of this cascade, however it helps to have the total map first.

Parsing high quality is a multi-check choice. The pipeline begins low cost and asks, at every verify, whether or not the parser produced sufficient for the query. Each verify has a price; each verify that flags an issue can route the affected web page to a deeper parser. The pipeline pays just for the standard the query wanted.

The pipeline’s 4 bricks (doc parsing, query parsing, retrieval, era, launched in Articles 5 to eight) every personal one to 3 of the 9 analysis checks. The grouping makes the cascade legible: each verify belongs to a brick, each brick has its personal price profile and its personal attribute verify. The cascade diagram in part 1 reveals the total mapping.

The 2 recurring examples make the cascade concrete.

• Desk 3 on web page 9 of the Consideration paper is a flat-parsed grid: each cell by itself line with no column anchor surviving. Textual content density alone misses it at verify 1; the deterministic check-2 fingerprint catches it cleanly (flat-table signature), and verify 7 (the LLM flag) confirms it, with the caveats developed within the second half.
• Determine 1 on web page 3 is a diagram physique that PyMuPDF returns empty. Catchable at verify 1 (lower-than-average textual content density mixed with an embedded picture), at verify 2 (opaque determine area with zero extracted chars), and at verify 7.

The most affordable verify that fires wins, and the pipeline by no means asks the slower checks a query the quicker checks already answered.

The info mannequin that helps it: No matter verify triggers escalation, the schema for the rerun is identical. Add one column, parsing_method, to the relational tables from Article 5. The escalation logic turns into: write new rows with a deeper methodology on the affected pages. Blended paperwork, audit path, and caching all fall out of the schema at no cost.

page_df is the canonical place as a result of parsing all the time occurs per web page. After a typical adaptive run on the Transformer paper, the place the cascade flagged web page 6 (Desk 1, Most path lengths, caught by the check-2 flat-table fingerprint) and web page 3 (Determine 1, caught by the char-density sign at verify 1 and the opaque-figure sign at verify 2), page_df appears to be like like this. This part makes use of Desk 1 to indicate the form of the escalated tables; the total end-to-end walkthrough within the second half takes a second flat desk, Desk 3 on web page 9 (Variations on the Transformer structure), so that you see the sample on two completely different actual tables slightly than one.

Two design choices are baked in. First, escalation provides new rows, it doesn’t exchange. The PyMuPDF row for web page 6 stays alongside the Azure row: “PyMuPDF tried, the cascade flagged the parse, Azure was referred to as and succeeded”. Second, the context_structured column on page_df lets downstream queries choose the proper row: SELECT * FROM page_df WHERE page_num=6 AND context_structured=True returns the trusted parse with none re-run.

line_df additionally carries parsing_method. Strains from PyMuPDF coexist with traces from the deeper parser on the identical web page. PyMuPDF produced the prose round Desk 1; Azure produced the markdown rows of the desk itself; each keep in line_df. The retrieval brick reads line_df and sees a uniform form, no matter which parser produced which row.

Pictures reside in line_df too, as rows with kind='picture' and a placeholder textual content. That is the bit that makes the determine case (within the second half) work mechanically the identical because the desk case. The opaque-figure sign at verify 2 already flags such pages from line_df alone (picture current, zero extractable chars inside its bbox). When the cascade flags web page 3, the pipeline picks the row from image_df, sees parsing_method='not_parsed', and calls a imaginative and prescient LLM. The imaginative and prescient output will get appended to line_df as new textual content rows with parsing_method='vision_gpt4o', and the image_df row is up to date.

Focused re-parsing is one perform name, method-agnostic, that returns new rows with parsing_method=methodology already populated:

def enrich(pdf_path, line_df, page_df, pages, methodology):
    """Add layer-2 rows for `pages` utilizing `methodology` ; maintain current rows for audit."""
    if methodology == "azure_layout":
        from docintel.parsing.pdf.azure_layout import azure_layout_pdf_to_line_df
        new_lines = azure_layout_pdf_to_line_df(pdf_path, pages=pages)
    elif methodology == "vision_gpt4o":
        new_lines = vision_extract(pdf_path, pages=pages)
    # new_lines already carries parsing_method=methodology
    line_df = pd.concat([line_df, new_lines], ignore_index=True)
    page_df = append_page_rows(page_df, pages, methodology)
    return line_df, page_df

The line_df schema from Article 5 is unchanged for downstream bricks: retrieval and era learn line_df, group by page_num, and by no means have to know which parser produced which line. The one contract that modified is “the parser identification travels with the row”.

Doc parsing owns three of the 9 checks: pre-parsing (verify 1, earlier than any extraction runs), parsing-time (verify 2, on PyMuPDF’s personal outputs), and post-parsing (verify 3, on chunks downstream of line_df). All three are deterministic, free, and produce a verdict in milliseconds with none LLM name. Collectively they catch many of the failure shapes the article cares about, together with each Case A (the flat desk, Desk 3 on web page 9) and Case B (the opaque determine, Determine 1 on web page 3), the 2 circumstances the second half walks finish to finish.

The most affordable analysis level sits earlier than any text-extraction name runs. PyMuPDF can already report per-page metadata in milliseconds: what number of characters it might extract, what number of embedded photographs every web page carries, what producer string the PDF declares. These indicators route many of the corpus earlier than the parsing choice turns into attention-grabbing.

A local-PDF web page usually extracts 2000-4000 characters of textual content. Pages that are available effectively under that, paired with a number of embedded photographs, are signalling that “the web page is generally a determine” with none deeper inspection.

For Determine 1 on web page 3, this sign alone is sufficient to flag: 1827 chars vs the doc’s 2633-char imply, plus one embedded picture. The pipeline may route web page 3 to a imaginative and prescient LLM with out ever calling Azure or operating a era cross. For Desk 3 on web page 9, the identical sign doesn’t hearth: 2973 chars (above imply), zero embedded photographs. Test 1 catches one of many two examples at no cost; verify 2 will catch the opposite.

The PDF metadata provides a second low cost classifier. The Consideration paper’s producer is pdfTeX-1.40.25, which says LaTeX-authored, native, no OCR wanted. A scanner-produced PDF would say Adobe Scan, Canon, OmniPage. That single string routes 80% of an enterprise corpus to the proper tier earlier than any textual content extraction runs. The creation date, the embedded font checklist, and the presence of an XMP block add coarse however helpful classifiers.

pre = pre_parse_signals("knowledge/paper/1706.03762v7.pdf")

pre["producer"]            # -> "pdfTeX-1.40.25"
pre["is_scanner_output"]   # -> False
pre["pages"].head()
# page_num  char_count  image_count  image_bboxes
#        1        2858            0  []
#        2        4256            0  []
#        3        1827            1  [(196.6, 72.0, 415.4, 394.4)]   <-- Determine 1
#        4        2508            2  [(175.0, 94.0, 239.0, 221.3),
#                                     (346.8, 82.7, 467.0, 267.3)]   <-- Determine 2
#        5        3193            0  []

The results of this verify is just not the reply to a query. It’s a per-page judgment: “this web page is regular”, “this web page has a lacking picture physique”, “this entire doc is scanned and desires OCR upfront”. The judgment is free and it ships with the parsing_method column on page_df.

PyMuPDF runs subsequent. Its outputs already carry deterministic indicators in regards to the structure it simply walked. Three checks hearth from line_df alone, none of them needing an LLM name.

PyMuPDF extracts each line with bounding-box coordinates. Clustering line x-coordinates per web page produces a column depend. A well-structured doc has a secure column depend from web page to web page. Variance is the routing sign.

The anomaly on web page 10 is itself a parsing-quality sign. The detector didn’t produce a improper depend accidentally, it produced it as a result of the desk broke the column mannequin. That’s info the pipeline can use to route web page 10 to a layout-aware parser.

When PyMuPDF flattens a desk, the cells fall on their very own traces with slim bboxes instantly following the “Desk N: …” caption.

The fingerprint is concrete: on web page 9, traces 5-20 maintain the desk headers cut up throughout two bodily rows of bboxes, and features 21-33 maintain the 13 cells of the base row every by itself line. The cells are brief (1-4 chars), slim (below 30 PDF factors vast), and clustered vertically in a decent band proper after a “Desk 3: …” caption. A verify that counts these properties on line_df[page_num=9] and journeys a boolean flag runs in O(n_lines) with none LLM name. Desk 3 is caught at this verify.

PyMuPDF experiences picture bboxes at extraction time by way of web page.get_image_info(). If a web page has a picture and the textual content extracted inside its bbox is empty, the determine physique didn’t survive the parse.

The verify is identical because the pre-parsing density verify however extra exact: as a substitute of noticing the web page has fewer characters, it pinpoints a particular rectangle with zero characters. Determine 1 is caught at this verify. (Pre-parsing additionally catches it, extra cheaply; verify 2 confirms it and provides the bbox the following parser will want.)

The Consideration paper is single-column, so it can’t illustrate this verify. We swap in BERT (1810.04805) which is the usual ACL two-column format. assign_column_positions clusters line x-coordinates per web page and labels every line “left” / “proper” / “single” / “multi”. No hardcoded midpoint, the cut up is computed from the precise line distribution. When PyMuPDF walks such a web page geometrically, it might interleave sentences throughout the column boundary; the column labels let the pipeline (or a deeper parser) restore the proper studying order.

pre     = pre_parse_signals("knowledge/paper/1706.03762v7.pdf")
line_df = fitz_pdf_to_line_df("knowledge/paper/1706.03762v7.pdf")

# Three pages, three indicators, no LLM name
page_level_parsing_signals(line_df, page_num=3, pre_signals=pre)
# {'page_num': 3, 'flat_table': False, 'opaque_figure': True,
#  'col_anomaly': False, 'causes': ['opaque_figure'], 'escalate': True}

page_level_parsing_signals(line_df, page_num=9, pre_signals=pre)
# {'page_num': 9, 'flat_table': True, 'opaque_figure': False,
#  'col_anomaly': False, 'causes': ['flat_table'], 'escalate': True}

page_level_parsing_signals(line_df, page_num=10, pre_signals=pre)
# {'page_num': 10, 'flat_table': True, 'opaque_figure': False,
#  'col_anomaly': True, 'causes': ['flat_table', 'col_anomaly_3'],
#  'escalate': True}

On the finish of verify 2, each Desk 3 and Determine 1 are flagged for escalation with no single LLM name. The pipeline may enrich each pages with the proper deeper parser now, earlier than retrieval even begins. The remaining checks of the pipeline (retrieval, era, post-gen) grow to be last-resort security nets, not the first detection mechanism.

After line_df is constructed, the pipeline chunks it for retrieval. A piece that cuts mid-sentence or mid-table-row carries a low chunk_integrity_score that downstream prompts can learn. Surya and PaddleStructure expose this instantly; with line_df alone we will compute it from line numbers and bboxes (sentence-end punctuation within the final line, table-row alignment inside the chunk).

The rating is deterministic, low cost, and fires earlier than any retrieval name. The article doesn’t develop a labored instance for verify 3 as a result of each Case A and Case B are already caught at checks 1 and a couple of; we point out it right here so the panorama is full.

Query parsing turns the query right into a typed parsed_question (Article 6) and makes use of that to route the cascade. Two checks reside right here: verify 4 (intent routing, one small LLM name on the query) and verify 9 (person suggestions, a gradual human-driven loop that refines verify 4 over time).

“In Desk 3”, “what share”, “in keeping with article 5” sign that the reply lives in a structured area. Route on to table-aware retrieval, skip prose embeddings. Article 6 made the case for parsing the query right into a typed object; routing by intent is certainly one of its payoffs. The price of this verify is one small LLM name on the query (just a few cents per thousand questions). The standard acquire: the pipeline routes Desk 3 inquiries to a table-aware parser with out ever needing the LLM at verify 7 to note the desk was flattened.

Test 9 is the long-loop verify: a query that the person re-asks with barely completely different wording, a golden set drift over time, an knowledgeable flagging a improper reply. None of those hearth inside a single pipeline run, however they refine the query parser (and, not directly, each verify that follows). Concretely: a recurring rewording captured in expert_keywords_df (Article 6) will get added to the query router, and the following query that makes use of that phrasing is appropriately categorized with out the person having to retype it. The price of this verify is human time spent on golden-set curation; the standard acquire compounds.

Retrieval takes the parsed query and the parsed corpus, and returns a small set of candidate chunks. Two checks hearth right here, each deterministic and each operating on portions the retrieval brick already computed.

High-1 cosine of 0.45 is a poor anchor. High-1 of 0.85 with top-2 at 0.83 is ambiguity. Each deterministic, each hearth earlier than era. Reranker disagreement with the bi-encoder is itself a second-order sign. The price is the cosine computation already completed by retrieval; the standard acquire is catching questions that landed nowhere clear.

As soon as the candidate chunks are picked, a final deterministic verify compares them to the query’s parsed scope. Query covers 2020-2024, chunks cowl 2020-2022: the hole is detectable earlier than the LLM name. Identical for chunks that contradict one another on a key truth. These checks run in milliseconds and are skipped solely at the price of fabrication-prone solutions downstream.

The identical cheap-then-rich logic the cascade applies to parsing additionally applies to embedding. Parsing produced page_df and line_df (Article 5 Part 3.2 mentioned “line_df retains the door open” on this); retrieval now decides which granularity to embed, and when.

The cascade turns parsing high quality right into a sequence of low cost questions. Pre-parsing metadata routes many of the corpus earlier than any extraction runs. Parsing-time fingerprints flag flat tables and opaque figures from PyMuPDF’s personal output. Intent routing and the retrieval score-gap catch circumstances the parser’s output alone can’t. Each verify is deterministic, free, and runs in milliseconds, and each one that fires can route a single web page to a deeper parser.

One column, parsing_method, makes the entire thing auditable: escalation provides rows, it by no means overwrites, so a web page can carry its PyMuPDF parse and its Azure parse facet by facet. What none of those low cost checks can catch is the failure that solely reveals up when the LLM tries to make use of the textual content. That final line of defence, and the 2 escalations walked finish to finish (a flat desk to Azure, a determine to a imaginative and prescient LLM), are the second half: Loop engineering with adaptive parsing in motion: parsing flat tables with Azure and figures with a imaginative and prescient LLM (hyperlink to return).