ocr-extraction.md 8.5 KB

OCR & Document Text Extraction (merged from the ocr-and-documents skill)

Scripts referenced below live in this skill's scripts/ directory.

PDF & Document Extraction

For DOCX: see the microsoft-office skill (create/edit) or use python-docx for structured reads. For PPTX: see the microsoft-office skill (full create/read/edit support). For PDF manipulation (merge, split, forms, watermarks, creation): see the pdf skill. This skill covers text extraction from PDFs and scanned documents.

Coming from a read_file EXTRACTION COVERAGE WARNING? read_file auto-converts local PDFs but reads the text layer only; the warning footer lists the pages that yielded no text (scanned images). For a handful of pages, render + vision is fastest: pdftoppm -jpeg -r 150 -f N -l N file.pdf $TMPDIR/page then vision_analyze each image. For bulk OCR of many pages, use marker-pdf below (Step 2).

Step 1: Remote URL Available?

If the document has a URL, always try web_extract first:

web_extract(urls=["https://arxiv.org/pdf/2402.03300"])
web_extract(urls=["https://example.com/report.pdf"])

This handles PDF-to-markdown conversion via Firecrawl with no local dependencies.

Only use local extraction when: the file is local, web_extract fails, or you need batch processing.

Step 2: Choose Local Extractor

Feature pymupdf (~25MB) marker-pdf (~3-5GB)
Text-based PDF ✅ ✅
Scanned PDF (OCR) ❌ ✅ (90+ languages)
Tables ✅ (basic) ✅ (high accuracy)
Equations / LaTeX ❌ ✅
Code blocks ❌ ✅
Forms ❌ ✅
Headers/footers removal ❌ ✅
Reading order detection ❌ ✅
Images extraction ✅ (embedded) ✅ (with context)
Images → text (OCR) ❌ ✅
EPUB ✅ ✅
Markdown output ✅ (via pymupdf4llm) ✅ (native, higher quality)
Install size ~25MB ~3-5GB (PyTorch + models)
Speed Instant ~1-14s/page (CPU), ~0.2s/page (GPU)

Decision: Use pymupdf unless you need OCR, equations, forms, or complex layout analysis.

If the user needs marker capabilities but the system lacks ~5GB free disk:

"This document needs OCR/advanced extraction (marker-pdf), which requires ~5GB for PyTorch and models. Your system has [X]GB free. Options: free up space, provide a URL so I can use web_extract, or I can try pymupdf which works for text-based PDFs but not scanned documents or equations."


pymupdf (lightweight)

Give it a separate side environment (~110 MB with its deps; never pip install into Hermes's own interpreter): Hermes builds it when hermes is on PATH, otherwise (a Docker sandbox without Hermes) a venv in the sandbox home takes the same requirements. Then run the helper and the inline snippet below with "$PY" instead of python:

if command -v hermes >/dev/null 2>&1; then
  PY="$(hermes pm environment pdf-pymupdf --requirement 'pymupdf>=1.26,<3' --requirement 'pymupdf4llm>=0.2,<3')"
else
  PY="$HOME/.cache/hermes-pdf-pymupdf-venv/bin/python"
  "$PY" -c "import pymupdf, pymupdf4llm" 2>/dev/null || (
    # concurrency-safe build: see the documents block in SKILL.md
    mkdir -p "$HOME/.cache" && cd "$HOME/.cache" && stage="$(mktemp -d hermes-pdf-pymupdf-venv.XXXXXX)" || exit 1
    old="$(readlink hermes-pdf-pymupdf-venv)"  # the published build, if any (another agent may have just published it)
    python3 -m venv "$stage" &&
      "$stage/bin/python" -m pip install -q --disable-pip-version-check 'pymupdf>=1.26,<3' 'pymupdf4llm>=0.2,<3' &&
      "$stage/bin/python" -c "import pymupdf, pymupdf4llm" &&
      if [ -z "$old" ]; then python3 -c 'import os, sys; os.symlink(*sys.argv[1:])' "$stage" hermes-pdf-pymupdf-venv
      elif [ "$(readlink hermes-pdf-pymupdf-venv)" = "$old" ] && ! "$old/bin/python" -c "import pymupdf, pymupdf4llm" 2>/dev/null; then
        ln -s "$stage" "$stage.lnk" && python3 -c 'import os, sys; os.replace(*sys.argv[1:])' "$stage.lnk" hermes-pdf-pymupdf-venv &&
          case "$old" in hermes-pdf-pymupdf-venv.??????) rm -rf "$old" ;; esac
      fi 2>/dev/null
    [ "$(readlink hermes-pdf-pymupdf-venv)" = "$stage" ] || rm -rf "$stage" "$stage.lnk"
  )
fi
export PY

Via helper script:

"$PY" scripts/extract_pymupdf.py document.pdf              # Plain text
"$PY" scripts/extract_pymupdf.py document.pdf --markdown    # Markdown
"$PY" scripts/extract_pymupdf.py document.pdf --tables      # Tables
"$PY" scripts/extract_pymupdf.py document.pdf --images out/ # Extract images
"$PY" scripts/extract_pymupdf.py document.pdf --metadata    # Title, author, pages
"$PY" scripts/extract_pymupdf.py document.pdf --pages 0-4   # Specific pages

Inline:

"$PY" -c "
import pymupdf
doc = pymupdf.open('document.pdf')
for page in doc:
    print(page.get_text())
"

marker-pdf (high-quality OCR)

# Check disk space first
python scripts/extract_marker.py --check

# its own side environment (PyTorch + models); run the helper with "$PY"
if command -v hermes >/dev/null 2>&1; then
  PY="$(hermes pm environment pdf-marker --requirement 'marker-pdf>=1.8,<3')"
else
  PY="$HOME/.cache/hermes-pdf-marker-venv/bin/python"
  "$PY" -c "import marker" 2>/dev/null || (
    # concurrency-safe build: see the documents block in SKILL.md
    mkdir -p "$HOME/.cache" && cd "$HOME/.cache" && stage="$(mktemp -d hermes-pdf-marker-venv.XXXXXX)" || exit 1
    old="$(readlink hermes-pdf-marker-venv)"  # the published build, if any (another agent may have just published it)
    python3 -m venv "$stage" &&
      "$stage/bin/python" -m pip install -q --disable-pip-version-check 'marker-pdf>=1.8,<3' &&
      "$stage/bin/python" -c "import marker" &&
      if [ -z "$old" ]; then python3 -c 'import os, sys; os.symlink(*sys.argv[1:])' "$stage" hermes-pdf-marker-venv
      elif [ "$(readlink hermes-pdf-marker-venv)" = "$old" ] && ! "$old/bin/python" -c "import marker" 2>/dev/null; then
        ln -s "$stage" "$stage.lnk" && python3 -c 'import os, sys; os.replace(*sys.argv[1:])' "$stage.lnk" hermes-pdf-marker-venv &&
          case "$old" in hermes-pdf-marker-venv.??????) rm -rf "$old" ;; esac
      fi 2>/dev/null
    [ "$(readlink hermes-pdf-marker-venv)" = "$stage" ] || rm -rf "$stage" "$stage.lnk"
  )
fi
export PY

Via helper script:

"$PY" scripts/extract_marker.py document.pdf                # Markdown
"$PY" scripts/extract_marker.py document.pdf --json         # JSON with metadata
"$PY" scripts/extract_marker.py document.pdf --output_dir out/  # Save images
"$PY" scripts/extract_marker.py scanned.pdf                 # Scanned PDF (OCR)
"$PY" scripts/extract_marker.py document.pdf --use_llm      # LLM-boosted accuracy

CLI (installed with marker-pdf next to "$PY", not on PATH):

"$(dirname "$PY")/marker_single" document.pdf --output_dir ./output
"$(dirname "$PY")/marker" /path/to/folder --workers 4    # Batch

Arxiv Papers

# Abstract only (fast)
web_extract(urls=["https://arxiv.org/abs/2402.03300"])

# Full paper
web_extract(urls=["https://arxiv.org/pdf/2402.03300"])

# Search
web_search(query="arxiv GRPO reinforcement learning 2026")

Split, Merge & Search

pymupdf handles these natively — use execute_code or inline Python:

# Split: extract pages 1-5 to a new PDF
import pymupdf
doc = pymupdf.open("report.pdf")
new = pymupdf.open()
for i in range(5):
    new.insert_pdf(doc, from_page=i, to_page=i)
new.save("pages_1-5.pdf")
# Merge multiple PDFs
import pymupdf
result = pymupdf.open()
for path in ["a.pdf", "b.pdf", "c.pdf"]:
    result.insert_pdf(pymupdf.open(path))
result.save("merged.pdf")
# Search for text across all pages
import pymupdf
doc = pymupdf.open("report.pdf")
for i, page in enumerate(doc):
    results = page.search_for("revenue")
    if results:
        print(f"Page {i+1}: {len(results)} match(es)")
        print(page.get_text("text"))

No extra dependencies needed — pymupdf covers split, merge, search, and text extraction in one package.


Notes

  • web_extract is always first choice for URLs
  • pymupdf is the safe default — instant, no models, works everywhere
  • marker-pdf is for OCR, scanned docs, equations, complex layouts — install only when needed
  • Both helper scripts accept --help for full usage
  • marker-pdf downloads ~2.5GB of models to ~/.cache/huggingface/ on first use
  • For Word docs: python-docx via the microsoft-office skill's "$PY" setup block (better than OCR — parses actual structure)
  • For PowerPoint: see the microsoft-office skill (uses python-pptx)