# pdfpeek

Most PDF tools dump text and leave you guessing whether it worked. You get a string back.
Whether that string is the document or a garbled phantom text layer over a scan is
something you find out later, downstream, usually in production.

pdfpeek returns the text **and a confidence score for every block.**

```python
from pdf_engine import extract

result = extract("document.pdf")
result.text          # extracted text
result.confidence    # 0.0 – 1.0, document level
result.warnings      # what went wrong along the way

for page in result.ir.pages:
    for block in page.blocks:
        print(block.text, block.confidence.final, block.block_type)
```

## The pipeline

Ten stages, and each one contributes to what the score ends up meaning.

| Stage | Name | What it does |
|-------|------|-------------|
| 0 | Triage | Classifies each page: text-native, scanned, or hybrid |
| 1 | Extraction | Pulls embedded text, with phantom-layer detection |
| 2 | Layout | Detects block types — heading, body, table, figure |
| 3 | Reading order | XY-cut partitioning, RTL-aware |
| 4 | Tables | Explicit (ruled) and implicit (whitespace) detection |
| 5 | OCR | Tesseract and surya for scanned and hybrid pages |
| 6 | Assembly | Merges the pymupdf and surya outputs, de-duplicates |
| 7 | Cross-page | Strips headers and footers, builds heading hierarchy |
| 8 | Post-processing | Rejoins hyphenation, corrects OCR errors |
| 9 | Confidence | Scores every block |

Stage 0 exists because the honest classification of a real-world PDF is often "both."
A scanned report with a text-native cover page will defeat any tool that decides once, at
the top, which kind of document it's looking at.

Stage 3 is RTL-aware because I needed it to be. Reading order in a right-to-left document
is not a mirrored version of the left-to-right case once columns and tables are involved.

## What the score is made of

Four dimensions, combined as a geometric mean rather than an average — so one bad
dimension drags the result down instead of being averaged away by three good ones.

| Dimension | Meaning |
|-----------|---------|
| `text_quality` | Fraction of printable, non-garbled characters |
| `method_score` | Trust in the extraction method — pymupdf 1.0, tesseract 0.7 |
| `order_quality` | Confidence in the reading-order placement |
| `type_quality` | Confidence in the block-type classification |

Above 0.8 at the document level means the extraction is reliable. Below 0.6 means check
it by hand or try the layout extra. The thresholds are documented rather than hidden,
because a number nobody can interpret is worse than no number.

## Tiered installs

The reason `unstructured` is a 5 GB dependency is that it ships the layout models whether
you need them or not. pdfpeek splits it:

```bash
pip install pdfpeek           # born-digital only, ~130 MB download
pip install pdfpeek[ocr]      # + Tesseract for scans, the binary comes from your OS
pip install pdfpeek[layout]   # + surya AI layout detection, ~5 GB
```

Most PDFs are born-digital. Most users should never download five gigabytes.

## Known limits, v0.1

Complex multi-column layouts — magazines, newspapers — can still get reading order wrong.
Non-English OCR error correction isn't implemented yet. Documents over 1,000 pages are
slow, because surya runs at two to three pages a second. Equations, handwriting and
table-in-sidebar structures are best-effort.

Written and published to PyPI in two days, MIT licensed, with a CLI and a Python API.
