Tables in PDFs for RAG: Don’t Flatten the Grid

The quantity you want sits in a desk, on the intersection of a row and a column. Flatten the PDF to textual content and that intersection is gone: the label lands in a single place, the worth in one other, and the mannequin is left guessing which quantity belongs to which row. Tables are the place naive parsing quietly loses the reply.

This text is a bonus in Enterprise Doc Intelligence, a sequence that builds an enterprise RAG system from 4 bricks. Tables in PDFs: a diagnostic and 5 composable operations that maintain the grid, as a substitute of a choice tree.

🧭 New to the sequence? Each article on this sequence sits on our two In direction of Knowledge Science writer pages, Angela Shi and Kezhan Shi. That’s the shortest solution to see what is roofed and the place this one sits.

📓 Runnable companion notebooks are on GitHub: doc-intel/notebooks-vol1.

The usual RAG pipeline reads a PDF, chunks it into textual content, embeds the chunks, and retrieves the closest match. The pipeline handles bullet factors and paragraphs tremendous. The second the reply to a query lives inside a desk cell, the usual pipeline begins hallucinating numbers, and infrequently no person notices till an auditor opens the supply doc.

This text is about why tables break enterprise RAG, and what to do as a substitute. It’s positioned as a bonus as a result of tables contact all 4 bricks (doc parsing, query parsing, retrieval, technology) and not one of the fundamental articles owns the subject alone. Article 5 mentions fitz.find_tables() with out a technique. Article 10 escalates adaptive parsing with out a table-specific cascade. Article 15 (making ready the corpus) extracts fields from tables with out saying retrieve from one. This bonus consolidates the fragments into one coherent therapy.

1. Why tables break the pipeline

A desk in a PDF is just not a desk within the information sense. It’s a set of rectangles drawn on a web page, with textual content positioned in cells, usually with out express row or column markers. The parser has to reconstruct the grid from spatial geometry. Typically it succeeds and also you get a clear DataFrame. Typically it doesn’t.

When it doesn’t, three issues go mistaken without delay. First, the row and column construction is misplaced, so the LLM downstream sees a stream of values with no relational that means. Second, the header is commonly solely on the primary web page of a multi-page desk, so pages two onward develop into numerical noise. Third, the line-level quotation self-discipline of Article 8 breaks down as a result of there isn’t a solution to level at “row 47” when row 47 was by no means reconstructed as a row.

All three failures share a root. A desk is information that somebody put right into a format format as a result of the distribution format required it. The professional who created the schedule knew it was information. The parser that flattens the desk into textual content destroys what the producer of the doc already had. The precise transfer is to not deal with tables higher as textual content. It’s to revive them to their native structured kind as early as attainable and deal with them as information from then on.

2. 4 methods to symbolize a desk

The identical desk can dwell within the pipeline at 4 completely different ranges of construction. Selecting the correct degree for every desk is the primary design determination, earlier than any operation runs. The 4 ranges are usually not alternate options to argue between within the summary. Every is the appropriate reply for a selected mixture of desk measurement, schema stability, and query form.

A. Row-as-line in line_df is the default. It’s what the parser produces naturally, and it’s sufficient for any query whose reply reads from the desk the identical method it reads from prose. Every row of the desk turns into one row of line_df with _type="desk" and the textual content rendered as a Markdown pipe row (| col1 | col2 | col3 |). The road retains its bounding field on the web page, so highlighting and quotation work the identical as for prose. Downstream bricks see “strains that occur to be Markdown-shaped” and by no means department on table-ness. That is what the Azure Doc Intelligence parser emits at the moment. A query like “what’s the deductible for property protection?” is answered from the Markdown row instantly, retrieved like every other line, cited like every other line. Most tables in mixed-content paperwork cease right here, as a result of no operation downstream wants to deal with the columns by identify.

B. Separate table_df. When it’s essential function on the 2D form, you elevate the desk out of line_df into its personal DataFrame, with column headers preserved as DataFrame columns and rows as DataFrame rows. The line_df retains a placeholder line pointing on the desk by id. Three operations require this: concatenating a desk that continues throughout 5 pages with the header solely on the primary, projecting to 2 columns out of fourteen as a result of the query asks about one 12 months, filtering to the rows whose area matches the scope. None of those are attainable at A. As soon as the desk is flattened into Markdown rows, the columns are textually seen however now not addressable.

C. Columnar extraction with named, typed columns. Some tables recur throughout paperwork in a steady form: an insurance coverage contract’s premium desk, a monetary assertion’s earnings abstract, a regulatory schedule with mounted fields. These are usually not “tables in a PDF” anymore. They’re information that the producer occurred to supply as PDF as a result of PDF was the distribution format, and the doc we learn is a format of knowledge that already had columns and kinds earlier than it turned a PDF. The precise transfer is to revive that authentic kind. Carry the tables right into a columnar retailer at ingestion time, listed by doc id and desk id. The selection of storage engine is orthogonal: Parquet on disk, DuckDB for in-process queries, Postgres if the corpus warrants an actual database. What issues is that columns are named (policy_number, start_date, premium_amount) and typed (date, decimal, varchar). As soon as that holds, the desk is information. You question it with SQL, you be a part of throughout paperwork, you mixture. C is what unlocks corpus-level questions: “what are the overall premiums throughout all my insurance coverage contracts?” requires that each contract’s premium column maps to the identical identify and kind, which is the assure C makes.

D. Columnar however heterogeneous. Typically you need corpus-level addressing however the tables resist a standard schema: completely different distributors, completely different variations, completely different layouts of “the identical” data. The content material lands in a single textual content column subsequent to its metadata (doc_id, web page, table_id). You retain document-level retrieval and full-text search throughout the corpus, however the 2D construction is gone. D is the sincere fallback when C’s preconditions are usually not met. It’s not often chosen proactively. It exhibits up when the staff tried for C and couldn’t normalize the schemas within the time out there.

The dispatcher picks one degree per desk. Most keep at A. A couple of escalate to B when a continuation or a projection is required. Those that recur throughout paperwork in a identified form get promoted to C at ingestion time. The orphan recurring tables fall to D.

The size that drive the selection are usually not mutually unique, which is why a linear determination tree fails. A local, well-parsed desk may also be very lengthy. A multi-page continuation may also dwell in a doc the place 80% of the quantity is tabular. A desk the parser failed on may also want column projection. Every actual desk sits on the intersection of three or 4 circumstances, and the appropriate reply is a small diagnostic per desk plus a handful of idempotent operations that transfer tables between ranges. That’s what the subsequent two sections describe.

3. The diagnostic: table_df_meta

For each desk detected in a doc, the diagnostic information 5 orthogonal properties. The result’s a small DataFrame (one row per desk) that the dispatcher reads to select the illustration degree (A, B, C, or D) and, when the extent is B, which operations to use.

Parse high quality: Three ranges. Excellent: the parser returned a clear grid with constant row and column counts (fitz.find_tables() on a local PDF with express desk borders is the everyday case). Partial: the parser returned cells however the grid is irregular (rows of various widths, lacking cells, merged cells misinterpreted), and the phrases have identified bounding packing containers on the web page. Failed: the parser discovered rectangles however no usable grid, the web page is scanned, or OCR returned textual content with out structural cues.

Measurement: The pair (n_rows, n_cols). The related threshold is “suits within the LLM context with the encircling query and immediate overhead”. Above the funds, projection (O3) turns into necessary; beneath, it’s non-obligatory. The precise cell depend is dependent upon the mannequin’s context window and the way a lot of it the prose and the system immediate already eat; no mounted threshold travels properly throughout deployments.

Header standing: Three values. Current: the primary row is detected as header (both by font weight, by border, or as a result of its cells comprise principally quick textual content whereas later rows comprise numbers). Absent: no row qualifies, actually because the producer relied on the first-page header to cowl the subsequent pages too. Continuation: the desk is a continuation of a earlier one and its actual header lives on an earlier web page.

Multi-page continuity: Three values. Autonomous: the desk begins and ends on the identical web page. Continued-from-N: similar column depend, similar column x-positions, no header on this web page, suggesting this can be a continuation of the desk on web page N. Continues-to-M: the desk on this web page ends on the backside and the subsequent web page begins with a desk of the identical column construction with no header. The detection rule is geometric (column positions match inside 5 pixels) plus a header test.

Doc-level context: The ratio of complete desk space to complete textual content space within the doc. The three instance paperwork in part 5 sit at 6%, 13%, and 26% respectively. A doc that crosses roughly half its physique space in tables is one the place the appropriate structure stops being RAG-on-text-with-tables-as-a-special-case and turns into SQL-on-extracted-tables-with-text-as-annotation. The precise crossover is qualitative, not numeric; the diagnostic experiences the ratio and the dispatcher reads it alongside the per-table fields.

These 5 columns of table_df_meta are impartial. A local, partial-quality, giant, headerless, multi-page desk in a table-dominant doc is a sound row. The dispatcher reads all 5 fields and composes the response.

4. 5 composable operations

Every operation takes a table_df (or a set of them) as enter and returns a remodeled table_df. Most keep inside degree B and produce a cleaner B; O4 is the one promotion from B to C (or D). They’re idempotent: making use of an operation that doesn’t match its precondition is a no-op, so the composition is protected.

O1. Structural reconstruction from positions. Applies when parse high quality is partial and phrase bounding packing containers can be found. The parser delivered cells however the grid is damaged. The operation rebuilds the grid by clustering phrase positions into column bands (x-coordinate histogram peaks) and into row bands (y-coordinate gaps wider than line top). The cells then snap to the (column, row) grid. This will recuperate a clear table_df that the native parser missed (the Commodity column in part 5’s CMO instance and the over-split overview within the NIST instance are each O1 candidates). Value: one geometric go per web page, negligible. Failure mode: irregular tables with merged cells throughout rows defeat the straightforward grid mannequin and set off O5.

O2. Multi-page concatenation with header propagation. Applies when multi-page continuity is continued-from-N or continues-to-M. The operation walks the consecutive tables, detects the run, copies the header from the primary desk of the run to the rows of the continuation tables, and emits a single concatenated table_df with a brand new column source_page to protect provenance. The geometric continuity test (similar column depend, similar x-positions inside tolerance, headerless continuation) is the precondition. With out this operation, a 200-row schedule cut up throughout 8 pages produces 8 separate table_df objects of which 7 are semantically orphan.

O3. Query-driven projection. Applies when measurement exceeds the context threshold. Reads the query’s scope_filters and concept_keywords (Article 6) and tasks the desk to the columns whose headers match the query ideas, then filters the rows whose values match the scope filters. A 200-row, 20-column schedule requested “what’s the premium for property protection in California” will get projected to columns [coverage_type, state, premium] and filtered to rows the place state = 'CA', returning possibly 4 rows to ship to the LLM. That is the “filter earlier than retrieval” of Article 17 (querying the corpus), utilized on the desk degree inside one doc.

O4. Columnar extraction (B → C, or B → D). Applies when the desk recurs throughout paperwork in a identified form, or when the doc is dominated by tables that share a schema. The operation lifts the desk(s) into the columnar retailer described in part 2 (degree C), listed by doc id and desk id. Subsequent questions are dispatched to a SQL agent (the sample of Article 17, querying the corpus) as a substitute of the retrieve-and-generate pipeline: the LLM writes SQL, the engine executes, the LLM interprets the outcome. The promotion succeeds at degree C when a standard schema will be recognized throughout the matching tables; it lands at D when the schemas resist normalization.

O5. Imaginative and prescient-LLM fallback. Applies when O1 has failed (or when parse high quality is failed from the beginning). The operation renders the web page area across the desk as a picture and sends it to a vision-capable LLM with a structured immediate asking for a JSON illustration of the desk. Value: at the very least an order of magnitude costlier than O1, which is a free native geometric go. The precise value is dependent upon the mannequin and the picture measurement, however the hole is at all times extensive sufficient that O5 should keep a fallback, not a default, or the associated fee compounds rapidly throughout lengthy paperwork.

The operations compose. A desk that’s partial + continued-from-3 + giant + query requires filtering runs O1 then O2 then O3. A doc that’s table-dominant with continued tables runs O2 throughout all continuations, then O4 on the consolidated tables. The dispatcher picks the composition.

5. The dispatcher

The dispatcher reads table_df_meta and emits a sequence of operations per desk (or per doc, for O4). It’s deterministic and sufficiently small to slot in one Python file, in line with the resolve.py sample of Article 13.

Earlier than the three examples, one preliminary level that sits beneath all of them: the parser selection issues at the very least as a lot because the operation selection. The identical web page parsed by two completely different instruments can land in very completely different diagnostic buckets, so the dispatcher’s first job is to resolve which parser to invoke. Solely then does the per-table operation composition are available in.

Take Desk 3 of Consideration Is All You Want (Vaswani et al. 2017; arXiv non-exclusive distribution license, declared on the arXiv summary web page; information/paper/1706.03762v7.pdf, web page 9). The desk is a hyperparameter-ablation grid with round twenty rows (header, base config, the (A)-(E) variation blocks, the huge config) and 13 columns (the variation label, the structure hyperparameters N / d_model / d_ff / h / d_k / d_v, the regularization hyperparameters P_drop / ε_ls, prepare steps, and the metrics PPL / BLEU / params). It’s precisely the form of outcome desk the article ought to reward the reader with. The article additionally parses World Financial institution CMO tables (CC BY 3.0 IGO) and makes use of Azure Doc Intelligence (proprietary, Microsoft’s On-line Companies Phrases) as one of many parsers.

Azure Doc Intelligence on the identical web page recovers all 13 columns cleanly, together with the sparse cells:

The diagnostic reads radically completely different on the identical web page. With Fitz: failed parse high quality (Desk 3 unrecoverable with out O1+O5). With Azure DI: excellent. The Fitz pipeline has to do actual work to salvage something; the Azure pipeline will get clear table_df free of charge and skips O1-O5 fully. Selecting the correct parser up entrance absorbs work the operations in any other case need to redo, and Azure’s per-page value is small in comparison with a mistaken reply on a hyperparameter-ablation query.

That is the spirit of Article 10’s adaptive escalation, restricted to the desk case. Begin with a budget parser. When the diagnostic flags a tough desk, escalate that desk (or that web page) to a stronger parser. The article 10 cascade and the B04 operation composition meet right here: the parser cascade decides what degree of table_df you begin from; the operations resolve what to do with it after.

Three examples on actual public-domain paperwork. The scan outcomes beneath come from a Fitz find_tables() sweep over each web page; the parsed-table snapshots are produced by quick chunks on this article’s supply; each are reproducible.

Instance 1: NIST Cybersecurity Framework v1.1 (information/nist/NIST.CSWP.04162018.pdf, 55 pages, 28 tables discovered by Fitz, table-area ratio 26%). The Framework Core lives in Appendix A. Pages 30 to 32 maintain three abstract tables of reducing width: the 26 by 8 overview proven above the place Fitz inflates 4 logical columns into 8, then a 17 by 6 refinement on web page 31, then an 8 by 4 cut up on web page 32. From web page 33 onward the primary Core desk runs throughout roughly 20 pages with a steady 4-column form (Operate, Class, Subcategory, Informative References) and 6 to 9 rows per web page. Every continuation web page carries a header row that Fitz detects, however the precise Operate and Class values are clean on most rows as a result of the producer solely writes them as soon as on the high of every block.

The diagnostic reads: partial parse high quality on pages 30-32 (over-split grid), excellent on pages 33 onward, continued throughout the run, medium complete measurement, doc not table-dominant. Composition: [O1 on pages 30-32 to fold extra columns back to 4, O2 across pages 33-51 to forward-fill the missing Function and Category values and concatenate]. The result’s one table_df with the complete Framework Core (simply over 100 subcategories) joined on (page_num, table_id). With out O1 + O2, a query like “what’s subcategory PR.AC-3 about?” hits a continuation row the place Operate and Class are clean, and the LLM has no solution to know the subcategory belongs to Defend → Id Administration and Entry Management.

Instance 2: World Financial institution Commodity Markets Outlook, October 2025 (information/cmo/CMO-October-2025.pdf, 66 pages, 38 tables discovered by Fitz, table-area ratio 6%, two-column doc format). The Worth Forecasts desk sits on web page 17: 42 rows by 14 columns, one row per commodity, columns for historic years 2023-2024 and forecast years 2025-2027 throughout a number of revisions. Fitz captures the grid for the numeric block cleanly and utterly drops the Commodity column; each row’s first cell comes again empty. The diagnostic reads: partial parse high quality (label column lacking), autonomous (single web page), giant measurement, doc not table-dominant. Composition for a query like “what’s the wheat value forecast for 2027?”: [O1 to recover the Commodity labels by clustering the left-margin word positions into a column band, then O3 to project to the (commodity=wheat, year=2027) cell]. With out O1, the LLM sees 42 rows of decimal numbers underneath “2027f” with no concept which one is wheat; O3 alone on the damaged parse tasks to the appropriate 12 months however can’t decide the appropriate row.

The compositions are readable. An audit asking “why did this query return this row” walks the diagnostic, the operations utilized so as, and the ultimate table_df the LLM noticed. Each step is logged. There isn’t a hidden habits.

6. The query sort modulates the response

As soon as the appropriate table_df is in hand (after diagnostic and composition), the query form decides how the reply is produced. Three patterns.

Cell lookup: “What’s the premium for property protection in California?” The reply is one cell. The retrieval has filtered to 1 row; technology reads the cell and returns it with line-level quotation again to the supply web page and row. The annotated PDF (Article 1’s highlighting) is reused: the cited cell will get the rectangle.

Vary or column: “What are the deductibles for all protection sorts?” The reply is a column slice. The retrieval has projected to the related columns and will return the entire desk or a filter; technology returns the structured slice as a small markdown desk embedded within the reply schema. Quotation is on the desk degree fairly than the cell.

Mixture: “What’s the complete premium throughout all states?” The reply is a computation. The retrieval mustn’t have occurred on this department in any respect. The dispatcher routes the query to the SQL agent (Article 17, querying the corpus), which writes SELECT SUM(premium) FROM schedule WHERE ..., executes, and the LLM interprets the scalar outcome. Quotation is the SQL question plus the outcome, not a passage within the supply doc.

The query sort is the modulator that determines the reply’s form. It doesn’t change the diagnostic or the operations; it solely modifications what will get returned as soon as table_df is prepared.

7. What stays out of this text

A number of adjoining matters are actual, essential, and deliberately deferred to follow-up work to maintain this bonus centered.

Cross-document desk becoming a member of: “Examine the premium tables throughout all my insurance coverage contracts.” This requires schema alignment throughout paperwork (the contract-A column premium_amount and contract-B column prime_annuelle have to map). It’s corpus-level work, associated to the sector extraction of Article 15 (making ready the corpus) however utilized to whole tables fairly than scalar fields. Future sequence territory.

Purely visible tables: Bar charts offered as tables, color-coded matrices, infographic tables the place the cell worth is encoded by hue or measurement. These want vision-LLM therapy that goes past O5’s “reconstruct the grid” sample. They want semantic interpretation of visible encodings. Comply with-up work.

Complicated OCR on tables: Scanned tables with hand-written annotations, tables in non-Latin scripts, tables in paperwork the place the OCR layer is itself badly aligned with the visible layer. These want OCR-specific upstream work that’s its personal matter.

Lengthy structured varieties: Insurance coverage utility varieties, tax returns, regulatory disclosure varieties: these are form-shaped, not table-shaped. The excellence issues. A kind has named fields with values; a desk has rows of homogeneous construction. Kinds get area extraction (Article 15, making ready the corpus). Tables get the therapy on this article. A doc mixing each wants each remedies routed by the diagnostic.

8. Conclusion

The diagnostic-plus-composition sample is the appropriate response each time a parsing drawback is multi-dimensional and the scale are usually not mutually unique: a linear determination tree drops dimensions that didn’t make the minimize, a diagnostic plus composable operations offers each artefact the therapy its particular properties name for. The professional who constructed the desk knew it was information; the system’s job is to revive the desk to its native structured kind so the professional’s question lands on it. Dealing with tables as textual content, nevertheless rigorously, drops the row-and-column construction that the professional’s query is dependent upon.

9. Sources and additional studying

The vision-based table-structure mannequin behind most trendy desk extractors is Smock et al. (PubTables-1M / Desk Transformer, CVPR 2022). The top-to-end PDF pipeline with express TableFormer module is Auer et al. (Docling Technical Report, 2024). The layout-detection benchmark behind desk detectors is Pfitzmann et al. (DocLayNet, KDD 2022). The article’s framing: the diagnostic-plus-operations sample, table_df_meta measures desk properties (form, header construction, merged cells, density), and 5 composable operations dispatch on these properties to do actual work on the grid.

Earlier within the sequence:

• Doc Intelligence: sequence intro. What the sequence builds, brick by brick, and in what order.

What works, what breaks

• Baseline Enterprise RAG, from PDF to highlighted reply. The four-brick pipeline finish to finish: PDF in, highlighted reply out.

• Embeddings Aren’t Magic: The Predictable Failure Modes of RAG Retrieval. The place embedding similarity wins (synonyms, typos, paraphrase), the place it predictably breaks (unknown phrases, negation, term-vs-answer relevance), and use it anyway.

• RAG is just not machine studying, and the ML toolkit solves the mistaken drawback. Why chunk-size sweeps and finetuning optimize the mistaken factor; route by query sort as a substitute.

• From regex to imaginative and prescient fashions: which RAG method suits which drawback. Two axes, doc complexity and query management, that decide the method for every case.

  • 10 widespread RAG errors we maintain seeing in manufacturing. Ten manufacturing errors, organized brick by brick, with the repair for every.

Doc parsing

• Constructing Doc Construction with Loop Engineering: Recovering a PDF’s Define from Physique Typography for RAG. Rebuilding the define from physique typography when the PDF ships no contents web page in any respect: six alerts, one bounded loop.

• Earlier than Full Agentic RAG: Know How You Resolve, and the Parsing Strategies You Decide From. The parsing strategies as a list, and the choice of which to run, earlier than handing the loop to an agent.

Technology

• Most RAG Hallucinations Are Extraction Errors: Seven Patterns for a Typed Technology Contract. Seven recurring methods a mannequin will get the extraction mistaken, and the typed contract that catches each.

• Loop engineering for RAG technology: an LLM cascade from an inexpensive native mannequin as much as a hosted flagship. Beginning on an inexpensive native mannequin and escalating solely when the reply doesn’t maintain up, measured.

One-document pipelines

• Immediate Engineering Isn’t Sufficient: How 4 Bricks of Context Engineering Cease RAG Hallucinations. Why a greater immediate doesn’t repair a mistaken web page, and what every of the 4 bricks contributes to the context as a substitute.

• Minimize an Enterprise RAG Pipeline’s Latency and Value by Calling the LLM Much less, Not by Shopping for a Quicker Mannequin. Reducing a pipeline’s latency and value by calling the mannequin much less usually and cheaper, not by shopping for a quicker one.

• Loop engineering for cross-references: when RAG solutions ‘see Part 7.2’ as a substitute of the particular reply. When the reply says “see Part X”, the pipeline loops again and fetches it.

• RAG workflow and loop engineering: the dispatcher that decides when to loop and when to cease. Suggestions loops, bounded iteration, and the dispatcher, composed into one workflow.

  • Loop engineering for RAG: the small loops inside every step, the massive loops throughout the pipeline. The 2 scales of loop: small bounded loops inside every brick, huge generation-triggered loops throughout them.