Projects

pdfpeek

Repository · PyPI

Most PDF tools dump text and leave you guessing whether it worked. You get a string back. Whether that string is the document or a garbled phantom text layer over a scan is something you find out later, downstream, usually in production.

pdfpeek returns the text and a confidence score for every block.

from pdf_engine import extract

result = extract("document.pdf")
result.text          # extracted text
result.confidence    # 0.0 – 1.0, document level
result.warnings      # what went wrong along the way

for page in result.ir.pages:
    for block in page.blocks:
        print(block.text, block.confidence.final, block.block_type)

The pipeline

Ten stages, and each one contributes to what the score ends up meaning.

Stage Name What it does
0 Triage Classifies each page: text-native, scanned, or hybrid
1 Extraction Pulls embedded text, with phantom-layer detection
2 Layout Detects block types — heading, body, table, figure
3 Reading order XY-cut partitioning, RTL-aware
4 Tables Explicit (ruled) and implicit (whitespace) detection
5 OCR Tesseract and surya for scanned and hybrid pages
6 Assembly Merges the pymupdf and surya outputs, de-duplicates
7 Cross-page Strips headers and footers, builds heading hierarchy
8 Post-processing Rejoins hyphenation, corrects OCR errors
9 Confidence Scores every block

Stage 0 exists because the honest classification of a real-world PDF is often “both.” A scanned report with a text-native cover page will defeat any tool that decides once, at the top, which kind of document it’s looking at.

Stage 3 is RTL-aware because I needed it to be. Reading order in a right-to-left document is not a mirrored version of the left-to-right case once columns and tables are involved.

scan012345678910 stagesheading 0.97body 0.91table 0.54example scores

What the score is made of

Four dimensions, combined as a geometric mean rather than an average — so one bad dimension drags the result down instead of being averaged away by three good ones.

text_qualitymethod_scoreorder_qualitytype_qualityaverage0.80geometric mean0.67example scores: one bad dimension drags it down
Dimension Meaning
text_quality Fraction of printable, non-garbled characters
method_score Trust in the extraction method — pymupdf 1.0, tesseract 0.7
order_quality Confidence in the reading-order placement
type_quality Confidence in the block-type classification

Above 0.8 at the document level means the extraction is reliable. Below 0.6 means check it by hand or try the layout extra. The thresholds are documented rather than hidden, because a number nobody can interpret is worse than no number.

Tiered installs

The reason unstructured is a 5 GB dependency is that it ships the layout models whether you need them or not. pdfpeek splits it:

pip install pdfpeek           # born-digital only, ~130 MB download
pip install pdfpeek[ocr]      # + Tesseract for scans, the binary comes from your OS
pip install pdfpeek[layout]   # + surya AI layout detection, ~5 GB

Most PDFs are born-digital. Most users should never download five gigabytes.

Known limits, v0.1

Complex multi-column layouts — magazines, newspapers — can still get reading order wrong. Non-English OCR error correction isn’t implemented yet. Documents over 1,000 pages are slow, because surya runs at two to three pages a second. Equations, handwriting and table-in-sidebar structures are best-effort.

Written and published to PyPI in two days, MIT licensed, with a CLI and a Python API.