Skip to content

Your model reads the PDF. The pages it never got are the problem

The big model APIs don't read your PDF as a picture: they send the text layer and the page image together. The real losses are quieter. The best open parsers pass about 83% of math checks, and scanned pages that never parse drop out of search without an error.

A claim is going around Russian-language tech feeds: language models only ever see a PDF as a flat picture, so you have to convert everything to Markdown first. And for math textbooks, the claim goes, nobody has solved the conversion, because the formulas always get lost.

The first half is wrong. The second half is out of date. Both pull attention away from where documents actually go missing.

The short answer

  • OpenAI and Anthropic both send the model the extracted text and an image of each page.
  • Open parsers do turn math PDFs into Markdown with LaTeX. On a public benchmark they pass about 83% of the math checks, not 100%.
  • The loss that hurts in production is quieter. Scanned pages that never parse don't raise an error. They just aren't in the index.

The strongest version of the claim

We start with the strongest version. Formulas in PDFs really do get mangled. A PDF stores glyphs placed on a page, not equations. A text layer can turn a fraction into three loose lines and a summation sign into a stray character. If your pipeline reads only that text layer, the math is gone. Anyone who has pasted a textbook page into a chat and got nonsense back has seen this happen.

So the instinct behind the claim is fair. The mechanism it describes is not.

What the APIs actually do

Both vendors document it.

OpenAI — File inputs (PDF, vision-capable models):
  "extracts both text and page images and sends both to the model"

Anthropic — PDF support:
  "The system converts each page of the document into an image.
   The text from each page is extracted and provided alongside
   each page's image."

The model gets two views of every page: the characters and the picture. A table border or the layout of a formula survives in the image even when the text layer scrambles it. "Only a picture" is close to the opposite of how this works.

Where formulas stand

Ai2's olmOCR 2 report (2025) scores parsers on olmOCR-Bench, a set of unit tests over real documents. Here are the math categories:

olmOCR-Bench (olmOCR 2 report, Ai2, 2025)

category                 olmOCR 2    Marker 1.10.1
ArXiv math               83.0        83.8
old scans with math      82.3        66.8

This table says two things. First, "nobody has solved this" is out of date. Specialized open parsers turn math into LaTeX inside Markdown, and on clean ArXiv papers these two are within a point of each other. Second, "solved" is wrong too. A pass rate of about 83% means roughly one formula check in six fails. On old scans the gap between the two tools opens to more than 15 points.

Our read: for a math textbook, the path that works is a specialized parser plus spot-checking the formulas by eye. "It's impossible" and "it just works" are both wrong.

Where documents really go missing

The formula debate is visible. You can look at a broken equation. The bigger loss is one you can't see.

One vendor wrote up a pilot at a fabric manufacturer: about 1,000 people, more than 125,000 SKUs, and decades of PDFs and scanned documents. The assistant came in below 60% accuracy. One of the causes was scanned PDFs that never parsed and quietly dropped out of retrieval. Nothing failed loudly. Those pages simply weren't there to be found. In the same discussion, another practitioner described a pipeline that reported success for weeks while whole documents were missing.

After the sources were cleaned up, the vendor reports accuracy of about 90%. Read that number with care. It is the vendor's claim, not our measurement, and the write-up does not isolate how much of the gain came from the scans. The failure mode is the part we trust, because it matches how ingestion pipelines are built: a parser that returns empty text has not technically failed.

A check you can run this evening

You don't need a new parser to find this. You need two counts and a comparison. Here is a sketch of ours:

# per document: pages in vs. pages that made it into the index

for doc in source_folder:
    pages_in       = page_count(doc)
    pages_parsed   = count(p for p in parse(doc)
                           if len(p.text.strip()) > MIN_CHARS)
    chunks_indexed = index.count(doc_id == doc.id)

    if pages_parsed < pages_in:
        flag(doc, "pages with no text: likely scans, send to OCR")
    if chunks_indexed == 0:
        flag(doc, "not in the index at all")

report:
    documents in folder     N
    documents in index      M
    pages with no text      K   (listed by file and page)

Three rules make that report worth reading:

1. empty text is a failure, not a success with zero results
2. count by document and by page: a document can be "in"
   with half its pages
3. run the report on every ingest, not once at launch

For math-heavy material, add a fourth step. Take a sample of formulas from each batch and compare them by eye against the page image. On the benchmark, roughly one check in six fails. Our rule of thumb is a few dozen formulas per textbook, which is enough to see where the errors cluster.

What would change our mind

If a major API documented that it sends PDFs as images only, the first half of this piece would be wrong, and we would correct it here. If an open parser scored close to 100% on math in olmOCR-Bench or a comparable public benchmark, we would drop the advice to proofread. And if the full pilot data showed that the missing scans made no difference to accuracy, the section on where documents go missing would need rewriting.

FAQ

Do I still need to convert PDFs to Markdown before sending them to a model? Not so the model can "see text". The big APIs already extract it. Converting still helps when you build your own search index, because then you control what gets indexed and you can check it.

Which parser should we use for math? On olmOCR-Bench, olmOCR 2 and Marker are within a point of each other on ArXiv math. On old scans, olmOCR 2 is ahead by more than 15 points. Our view: test both on twenty pages of your own material before you choose.

How do I know if scanned pages are missing from our search? Compare the page count of each file with the number of pages that produced text. Then compare the documents in the folder with the documents in the index. Any gap gives you a list of files to OCR.

Is 83% good enough for a textbook? Our view: for search, often yes. For anything a student copies a formula from, no. That is what the spot-check is for.

Related case

We work on a related problem in RE Intelligence. The DLD registry publishes millions of Dubai property transaction records in inconsistent formats, with Arabic names, duplicate areas and junk rows. In RE Intelligence every number comes from a deterministic query, source data is never silently "corrected", and the model only narrates what the data has already shown. The case is at iloblique.com/re-intelligence.

If you have something worth building, we'd like to hear about it.

Sources