Document Understanding and OCR

References

A synthesis of selected conference talks and technical references. Citations link to the source material; they do not imply that every talk on this subject is included.

1. OpenCV: Geometric Image Transformations

   OpenCV geometric transforms compute destination pixels from corresponding source locations. Affine and perspective transforms use matrices; remap accepts more general coordinate mappings. Fractional source positions require interpolation, and positions outside the image require a border policy. These mappings provide the mechanism for relating processed image regions to source coordinates.

2. PyMuPDF: OCR — Optical Character Recognition

   Optical character recognition recovers searchable text from images. PyMuPDF can recognize a complete document page or its image areas and retain the result in a TextPage for subsequent extraction. Its image-to-PDF procedure preserves the visible image while adding hidden recognized text. That text does not retain the original font, bold or italic styling. Suggested routing signals include an image covering the page, absent text, or many small graphics representing characters.

3. Microsoft Document Intelligence: Invoice data extraction

   Microsoft's invoice documentation distinguishes recognized text, tables and invoice-specific fields such as parties, totals and line items. It defines key-value pairs as source spans connecting a label with its associated value and notes that a detected key can have no associated value. Its inspection workflow exposes extracted fields alongside highlighted source text and detected tables.

4. Building AI Agents that actually automate Knowledge Work

   Incorrect processing of complex document layouts can undermine downstream reasoning regardless of model capability.

5. PROV-DM: The PROV Data Model

   PROV represents a source snapshot and its summary as distinct entities, with a summarization activity that used the snapshot and generated the summary. wasGeneratedBy can record generation time; wasDerivedFrom links the result to its source. Entity attributes can carry application-specific versions, and revision and invalidation relations describe later changes. Applied to retrieval, retain source identity/version, transformation identity, and separate source, retrieval, and summary-generation times. Freshness requires additional application policy: check the authoritative source's current version or validity, inspect invalidation, and enforce a task-appropriate age limit. Generating a summary today does not make its old source facts current.

6. Notes on transfer of responsibility for OCR-B standards

   Former OCR-B revision editor K. I. Larsson traces OCR-A and OCR-B to fonts developed for optical recognition during the 1960s in the United States and Europe. The two-part ISO 1073 published in 1976 specified these character sets and their shapes; ISO 1831, published in 1980, addressed printing tolerances. This documents an approach that constrained the printed input as well as specifying the characters to recognize.

7. An Overview of the Tesseract OCR Engine

   Ray Smith's 2007 account describes Tesseract's development at HP during 1984–1994, initially motivated by scanner applications and poor recognition of imperfect print. HP released it as open source in late 2005. The described engine groups connected image components into lines and words, handles fixed and proportional spacing differently, and recognizes words in two passes. Satisfactory first-pass words train an adaptive classifier; a second pass revisits uncertain words. Page-layout analysis was originally supplied separately.

8. Computer History Museum: Oral History of John Warnock, Part 2

   John Warnock recalls the early PDF effort as a document-interchange problem: application files and fonts did not transfer reliably between systems. Executing and linearizing PostScript produced explicit page boundaries and drawing operations without the original loops and conditionals. Fonts could be subsetted and packaged with the document. Warnock identifies Peter Hibbard and Richard Cohen as architects of the resulting format and dates his motivating Camelot paper to 1991.

9. Smithsonian: Farrington Character Sensing Machine 9SP3

   The Smithsonian's accession-based account dates David Shepard and Harvey Cook's Model No. 3 reader to late 1950 and Farrington's commercial sales to 1956. The preserved machine read numbers printed on 51-column tabulating cards and punched the recognized information into those cards. Its application was credit-sales invoice processing, connecting human-readable records to existing punched-card systems. The museum reports that it did not sell widely.

10. Kurzweil Computer Products: Ray Kurzweil's retrospective account

    Ray Kurzweil describes founding Kurzweil Computer Products in 1974 to recognize varied typefaces. By the end of 1975, the team combined omnifont recognition, a flatbed scanner and speech synthesis into a reading machine, announced in January 1976. Omnifont here means recognition across typefaces rather than requiring one specialized print style. The application let blind readers hear ordinary printed material. Kurzweil describes subsequent uses in database entry, word processing and document ingestion for Lexis and Nexis.

11. Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks

    CTC learns a distribution over output label sequences without requiring a manually aligned label for each input frame. At each step the network predicts labels plus a blank symbol; multiple frame-level paths collapse to the same shorter output after merging repetitions and removing blanks. Training sums over the alignments consistent with a target sequence. This addresses the mismatch between acoustic time steps and linguistic units, and distinguishes learning an alignment from simply assigning one word to each audio chunk.

12. Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks

    Alex Graves, Santiago Fernández, Faustino Gomez and Jürgen Schmidhuber introduced CTC at ICML 2006. Its original experiment evaluated phonetic labeling on TIMIT speech, comparing CTC with hidden Markov and hybrid systems. Handwriting appears as a motivating application, not as the reported experiment.

13. High-Performance OCR for Printed English and Fraktur using LSTM Networks

    Breuel and colleagues applied bidirectional LSTM networks to normalized text-line images in their 2013 printed-text OCR work. Connectionist Temporal Classification aligned transcripts with network outputs without requiring character-segmented training data. Their motivation was that incorrect character segmentation constrained earlier recognizers. Line normalization still mattered because vertical position and scale help distinguish characters. Published examples show recognition of touching characters and ligatures alongside errors involving unusual styles and decoding.

14. LayoutLM: Pre-training of Text and Layout for Document Image Understanding

    LayoutLM adds page-position information to textual representations so document models can use relationships that a plain word sequence omits. Its preprocessing obtains recognized words and their boxes through OCR, then scales coordinates to a shared range. The paper also incorporates image-region features and adapts the representation to form labeling, receipt extraction and document classification. For forms, it explicitly separates assigning semantic labels from linking entities, and evaluates the labeling task rather than semantic linking.

15. LayoutLM: Pre-training of Text and Layout for Document Image Understanding

    The KDD 2020 LayoutLM paper is by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei and Ming Zhou. Its architecture diagram combines pretrained text-and-layout representations with separately obtained image features for downstream tasks. This distinguishes the original model from later approaches that integrate image information during pretraining.

16. OCR-free Document Understanding Transformer

    Donut maps a document image through an encoder and generates a token sequence that can be converted into structured information. The paper's receipt illustration produces item records without consuming an external OCR result. The authors motivate this route partly through error propagation from separately supplied OCR. OCR-free therefore describes the absence of a separate OCR dependency, while the system still learns to read document content.

17. OCR-free Document Understanding Transformer

    Geewook Kim and the NAVER team presented Donut at ECCV 2022 following a November 2021 preprint. Its experiments show that larger input resolution particularly helps document questions involving tiny text, while increasing computation. The paper visualizes decoder attention as an auxiliary localization indicator without localization supervision. A published failure selects another visible number instead of the requested publication number, illustrating a field-association error despite producing a plausible transcription.

18. OCRmyPDF: Advanced features

    A PDF can mix born-digital text and scanned content. OCRmyPDF distinguishes skipping pages with text, replacing recognized invisible text, and rasterizing all content before recognition. Redo mode masks visible text while recognizing additional image text, but cannot distinguish OCR painted beneath an opaque image from ordinary visible text. Replacing text can invalidate existing structural tags; OCRmyPDF discards the structure tree when it cannot keep it aligned. A skipped page can remain in the output without OCR.

19. Error! The text copied from the PDF could not be pasted correctly

    A PDF can display characters correctly while copying or searching produces incorrect text. Callas explains two distinct lookups: selecting a glyph, the font-specific visible shape, and obtaining its Unicode representation. Its inspected example identifies disagreement between a ToUnicode mapping and font-encoding information. Successful rendering therefore does not establish that the embedded text can be extracted faithfully.

20. Microsoft: Structure of a WordprocessingML document

    WordprocessingML directly represents paragraphs, text runs and text content. A run groups text with shared properties such as formatting. Its document package can also contain separate parts for comments, footnotes, endnotes, headers, footers and styles. The main document body is therefore only one content region: extracting it alone need not recover all textual material in the file. This provides a native structural input that differs from recovering organization from page pixels.

21. OCRmyPDF: v17 release notes

    OCRmyPDF documents three distinct defects. Version 17.11 fixes redo processing that stacked another OCR layer because removal failed to inspect Form XObjects, embedded containers holding page content. Version 17.8 prevents PDF/A conversion from corrupting non-embedded CID font text layers: Ghostscript font substitution could destroy searchability; automatic mode preserves a regular PDF, while explicit PDF/A requests fail. Version 17.6 removes Adobe's embedded search index because document rewriting makes that index stale and can produce incorrect Acrobat search results.

22. Docling: Docling document

    DoclingDocument separates content items from document structure. Text, tables, pictures and key-value items reference parents and children through JSON pointers. Separate body and furniture trees distinguish main content from headers and footers. Child order within the body tree represents reading order. Items can carry bounding boxes and provenance, while tables can retain structure annotations.

23. hOCR 1.2: OCR Workflow and Output embedded in HTML

    hOCR embeds OCR structure and metadata in HTML. Its bounding box is a rectangle specified by left, top, right and bottom coordinates in pixels relative to the document image's top-left corner. Its content-flow property associates separate content areas belonging to the same flow. The specification recommends placing text in reading order and defines engine-specific confidence metadata.

24. Library of Congress: ALTO — Technical Metadata for Layout and Text Objects

    ALTO provides an XML schema for document layout and text metadata. Its version history documents explicit reading order using ordered groups, unordered groups and element references. Processing references can attach processing history to pages, blocks, lines and strings. Later additions describe page rotation, default language and text direction.

25. Structuring the Unstructured: Advanced Document Parsing for AI Workflows

    Docling combines OCR and layout analysis into a typed document representation that supports structural inspection and multiple exports, including extracted tables.

26. Tesseract: Improving the quality of the output

    Tesseract documents recognition failures from noise, uneven backgrounds, thick or thin characters, skew and unsuitable borders. Deskewing rotates a crooked scan so text lines become horizontal; excessive skew harms line segmentation. Recognition also depends on segmentation assumptions: a full page, line and isolated word need different treatment. Dictionaries suited to sentences can be inappropriate for codes or price lists.

27. PyMuPDF: Appendix 3 — Assorted Technical Information

    Coordinates locate content only relative to a defined origin and coordinate system. PyMuPDF distinguishes image pixels, whose origin is top-left, from page coordinates measured in points. Its documentation explains conversion between PDF and MuPDF spaces with transformation matrices and inverse mappings. It also shows a damaged PDF opening after repair, with warnings and a dirty document state rather than an ordinary untouched input.

28. PyMuPDF: Page

    PyMuPDF distinguishes the MediaBox from the visible CropBox and the page rectangle after rotation. Its renderer respects page rotation and clips to the intersection of the requested region and CropBox. Separate matrices translate PDF coordinates into MuPDF coordinates and convert between rotated and unrotated page space. For text highlights, the documentation recommends quadrilaterals because their ordered corners retain orientation information that a rectangle alone does not provide.

29. DewarpNet: Official implementation and OCR evaluation notes

    DewarpNet's authors disclose that their paper's OCR evaluation applied a blur operation before recognition and publish revised results without that operation. For the DewarpNet row, character error rate changes from 0.3136 with blur to 0.249 without blur. The documented evaluation uses 25 images, Tesseract 4.1.0 in its default configuration and PyTesseract 0.2.6. This supplies a controlled preprocessing comparison in which an additional image transformation worsened recognition.

30. An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition

    Baoguang Shi, Xiang Bai and Cong Yao's CRNN joins convolutional visual features, bidirectional recurrent sequence modeling and CTC transcription in one trainable recognizer. Image features become an ordered sequence of columns, allowing transcription without individual character boxes. Training uses image-level text sequences. The recognizer supports decoding with or without a lexicon, a prescribed set of permitted strings. Experiments demonstrate scene-word recognition using synthetic word images for training; results vary across datasets and lexicon settings.

31. Tesseract FAQ

    Tesseract's FAQ cautions that handwriting recognition works poorly because the engine is designed for printed text. It also distinguishes recognition from PDF-reader behavior: a searchable PDF appearing to contain only spaces can reflect a reader failing to understand its hidden text layer, rather than a defective PDF; the FAQ suggests checking another reader.

32. Building an Agentic Platform

    OCR errors contaminate the evidence passed to the model, creating extraction failures that model quality alone cannot resolve.

33. Docling Technical Report

    Docling's documented pipeline extracts native PDF text with coordinates and renders page images, applies layout and table models, then assembles a typed document with reading order and caption associations. Layout analysis predicts element regions and roles. TableFormer recovers rows, columns, spanning cells and header roles, including tables without complete borders; post-processing matches predicted cells to PDF text instead of retranscribing it.

34. Structuring the Unstructured: Advanced Document Parsing for AI Workflows

    Losing spatial structure can merge unrelated column text and flatten tables into text that no longer supports reliable extraction.

35. Recursive X-Y Cut using Bounding Boxes of Connected Components

    Ha, Haralick and Phillips's 1995 method recursively divides a page into rectangular regions at sufficiently large gaps in horizontal or vertical projection profiles. A projection profile summarizes where image components occur along an axis. Their implementation projects connected-component bounding boxes instead of individual pixels to reduce computation, producing a hierarchy of nested regions. The paper acknowledges that complex pages can leave several distinct content regions inside one unsplit leaf.

36. Empirical Performance Evaluation Methodology and Its Application to Page Segmentation Algorithms

    Song Mao and Tapas Kanungo's March 2001 study implemented both top-down X-Y cut and bottom-up Docstrum segmentation. Their account contrasts X-Y cut's rectangular, deskewed layout assumptions with Docstrum's ability to accommodate nonrectangular arrangements and arbitrary skew. It identifies Docstrum difficulties with nontext regions, irregular font sizes or spacing, and sparse characters. The study's own implementation omitted orientation estimation because its evaluation pages were already deskewed.

37. DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis

    Birgit Pfitzmann and colleagues at IBM introduced DocLayNet in 2022 to broaden layout coverage beyond scientific articles. It contains 80,863 manually annotated pages with 11 region classes. Their annotation trials found plausible disagreements about grouping subfigures and list items; explicit guidelines resolved some, while some ambiguity remained. A controlled split experiment also showed that mixing pages from the same documents across training and evaluation raised aggregate layout mAP from 72 to 84, compared with separating whole documents.

38. 2017 PAGE XML Format for Page Content

    PAGE XML records an image filename and dimensions, uniquely identified regions, polygon outlines, region types and text. Region types include pictures, tables, mathematics, noise and unknown content, allowing nontext content to remain represented. Regions can contain subregions, such as table cells or text inside figures. Reading order is stored separately through nested groups containing ordered or unordered region references. Text can be retained at region and line levels.

39. W3C Technique PDF3: Ensuring correct tab and reading order in PDF documents

    Reading order is the logical sequence consistent with a document's meaning. In tagged PDFs, tag order primarily determines this sequence. W3C illustrates a failure in which two columns are read across both columns instead of completing the first column before the second. Complex layouts containing footnotes, sidebars, graphics and tables can acquire incorrect order during conversion and require repair.

40. The Significance of Reading Order in Document Recognition and Its Evaluation

    Clausner and colleagues represent independent magazine articles as an unordered group while preserving paragraph sequences inside each article. Their 2013 evaluation compares relations between pairs of regions, including predecessor, successor and unordered relations. Because predicted segmentation can merge or split reference regions, it establishes region correspondences using overlap before assigning weighted penalties. This separates reading-order investigation from text-recognition accuracy and accommodates layouts without one required total sequence.

41. Unicode Standard Annex #9: Unicode Bidirectional Algorithm, Revision 51

    Unicode distinguishes logical character order in memory from visual display order. Arabic or Hebrew text can contain left-to-right digits and embedded Latin words, producing mixed directions within one line. The bidirectional algorithm determines display ordering from character properties and directional controls; the underlying characters remain interpreted in logical order. Consequently, traversing visible glyphs uniformly from left to right is not a general method for reconstructing the intended string.

42. Tagged PDF Best Practice Guide: Syntax 1.0.1

    Logical document structure need not restart at page boundaries. The guide represents a paragraph beginning on one page and continuing on another as one paragraph structure element linked to both page-content sequences. It also explains that an empty table-cell element can preserve meaningful table structure. Authoring mistakes can misrepresent paragraphs as tables or split one heading into multiple headings, so existing tags require interpretation and checking.

43. ICU4J: Edits.Iterator

    ICU's edit representation connects source-string spans and destination-string spans with separate start positions and lengths. It supports mapping indices in both directions. Within a changed span, exact internal correspondence may be unavailable; an interior index maps to the end of the corresponding edit. This provides a concrete mechanism for retaining span relationships when text transformations change length.

44. Google Document AI: Document REST resource

    Document AI separates document text, page layout and extracted entities. Text anchors reference one or more text segments; page anchors can span pages and multiple polygons. Entities separately expose mention text, normalized values and nested properties. Normalized values can be absent when conversion fails or is unsupported. Entity revisions can record successive extraction stages. The schema also describes OCR text replacements and revision ancestry.

45. W3C: Web Annotation Data Model

    A TextPositionSelector identifies a text span using an inclusive start and exclusive end. The specification requires its character-counting and normalization conventions to match TextQuoteSelector, which uses selected text plus surrounding context. Position selectors are brittle when a resource changes, so W3C recommends recording state to identify the correct representation. SVG selectors describe nonrectangular regions, including polygons, relative to the source image.

46. PyMuPDF: Document

    PyMuPDF distinguishes zero-based page indices from PDF page labels. Labels can use Roman numerals, prefixes or restarted numbering, and need not be unique. Its published example labels page index 6 as A-10 and page index 10 as 1. A label lookup therefore returns a list of matching pages rather than assuming one destination.

47. LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding

    Yang Xu and colleagues' LayoutLMv2 brings image features into pretraining alongside recognized text and layout. It adds relative spatial relationships, text-image alignment and text-image matching objectives. Alignment connects text lines to image regions; matching tests whether document text and imagery correspond. Its controlled DocVQA ablation found improved validation scores when adding visual information and the additional objectives, supporting their contribution beyond merely enlarging the input representation.

48. OpenRAG: An open-source stack for RAG — Phil Nash

    Docling offers focused extraction models or a task-specific vision-language model, while optional OCR and image descriptions add processing cost.

49. Forget RAG Pipelines—Build Production-Ready AI Agents in 15 Minutes

    Table misalignment and hallucinated chart extraction can corrupt the evidence supplied to every downstream stage.

50. IBM Granite-Docling-258M model card

    IBM's model card dates Granite-Docling-258M's release to September 17, 2025. It describes a specialized image-and-text model that generates DocTags for conversion into DoclingDocument and subsequent exports. Supported instructions include full-page conversion, recognition within a specified region, table conversion and document-element identification. IBM positions the model as a component of Docling that consolidates several specialized functions, rather than replacing the surrounding conversion library.

51. Gemini API: Document understanding

    Google documents different information paths for different file types. PDF processing supports visual document interpretation, while other accepted document types are processed as text without their rendered appearance. The Gemini 3 section additionally states that native PDF text is extracted and supplied to the model. Thus a document-model interface can combine embedded text with page imagery rather than relying exclusively on either representation.

52. Structuring the Unstructured: Advanced Document Parsing for AI Workflows

    The speaker describes frontier-model conversion as potentially accurate but costly at volume and vulnerable to output variation across runs and model versions.

53. PubTables-1M: Towards comprehensive table extraction from unstructured documents

    The paper separates table detection, row-column-cell structure recognition and functional analysis of keys and values. It identifies inconsistent splitting of spanning header cells as a ground-truth problem. Evaluation distinguishes exact content agreement in every cell from partial correctness. GriTS separately assesses cell topology, content and location while retaining two-dimensional ordering.

54. W3C WAI: Tables with Multi-Level Headers

    Complex tables can associate one data cell with several headers, including headers that change partway through the table. W3C represents these relationships using unique header IDs and explicit references from data cells. Its accommodation example associates a value with city, bedroom count and accommodation type, demonstrating why a cell's text alone does not express its meaning.

55. Microsoft Table Transformer: Official implementation

    Table Transformer recognizes table structure from images using object detection. Its extraction pipeline requires separately supplied text, obtained through OCR or native PDF extraction, to populate HTML or CSV output. The repository provides separate detection and structure-recognition models. Its release history distinguishes the initial project release in June 2021, full PubTables-1M release in October 2021, and detection and structure model weights released in March and May 2022.

56. NISO JATS: Table Wrapper Footer

    JATS represents explanatory table notes separately from tabular rows. Its published example contains a spanning header that supplies days as the unit for two subordinate columns. Elsewhere in the same table, the value 151/597 references a footnote stating that one trial's data were unavailable. The example makes both relationships explicit: units can come from a higher-level header, while a qualification can live outside the cell grid and be connected by an identifier.

57. Microsoft Document Intelligence: Custom template document model

    Template models use visual layout cues and suit highly structured documents with defined templates. Microsoft warns that template changes reduce accuracy. Its tabular-field guidance represents a table spanning pages as one table with rows labeled across those pages and calls for examples covering expected single-page and multipage variations. Repeated sections can also be represented as tabular fields.

58. Amazon Textract: Text Detection and Document Analysis Response Objects

    Textract represents documents as identified blocks connected by typed relationships. A page contains detected items, and a line references its words through CHILD relationships. VALUE relationships connect keys to values. Blocks can also carry confidence and page geometry. Thus containment and field association are represented separately.

59. DocILE Benchmark for Document Information Localization and Extraction

    DocILE distinguishes identifying and locating application fields from grouping those fields into line items, such as a product's description, quantity and price. Its localization metric checks field type and whether predicted regions capture the intended pseudo-character centers derived from OCR word boxes. Line-item scoring additionally checks assignment to the correct item. Dataset splits distinguish unseen layouts from layouts represented by few or many training examples, making template familiarity an explicit evaluation condition.

60. Amazon Textract: Selection Elements

    Textract models checkboxes, radio buttons and other selection marks as SELECTION_ELEMENT blocks with SELECTED or NOT_SELECTED status. Marks may occur in form fields or table cells. Its published service-rating table demonstrates that a selected mark must remain associated with both a row question and a response column.

61. Google Document AI samples: web-app-pix2info-python

    Google's published receipt example exposes different extraction operations. In one receipt, folded paper hides the supplier and mirrored text showing through the reverse side is incorrectly selected as its name. Another example retains the literal date 11/12/2022 while normalizing it to 2022-12-11 using French document context. A separate enrichment example supplies a complete address using external knowledge even though that address is not printed on the receipt. These distinguish observed text, contextual interpretation and externally added information.

62. Building an Agentic Platform

    Split large field sets into smaller tasks while keeping relationally dependent fields together.

63. OWASP Input Validation Cheat Sheet

    Syntactic validation checks the form of structured fields; semantic validation checks whether values make sense in their business context, such as a start date preceding an end date. JSON Schema is one supported validation mechanism. Validation applies to potentially untrusted backend feeds and supplier data as well as direct user input. OWASP explicitly cautions that input validation is not the primary defense against SQL injection or cross-site scripting.

64. Peppol BIS Billing 3.0: Rule BR-CO-13

    Peppol's BR-CO-13 requires the invoice total excluding VAT to equal the sum of invoice-line net amounts minus document-level allowances plus document-level charges. Its executable test compares supplied monetary fields and handles absent allowance or charge totals. This is a concrete internal-consistency check: the test operates on invoice values, without inspecting the page from which an extraction system obtained them.

65. Amazon Textract: GetDocumentAnalysis

    Textract's asynchronous analysis response separates returned blocks from job status, which can be IN_PROGRESS, SUCCEEDED, FAILED or PARTIAL_SUCCESS. Warning records contain an error code and affected page numbers. Separately, NextToken indicates that additional result blocks remain to be retrieved. The response also exposes the analysis model version. These fields distinguish incomplete processing from incomplete retrieval of an otherwise available result.

66. Building an Agentic Platform

    Box reports checking extracted answers against page images and, for some difficult questions, comparing answers from multiple vendors' models.

67. TEI P4 Guidelines: Transcription of Primary Sources

    The TEI guidelines distinguish material that cannot be transcribed, text transcribed with uncertainty and text supplied by an editor. A gap records unreadable or missing material without inserting a guessed reading; unclear marks a reading retained with uncertainty; supplied identifies an editorial replacement. The representation can record reasons, responsibility and the source of supplied text. Its examples preserve the difference between physical loss and a conjectured reconstruction.

68. Hugging Face Evaluate: Character Error Rate

    Character error rate is CER=(S+D+I)/N, where S replaces a reference character, D deletes one, I inserts an extra character, and N is reference length, equal to S+D+C for C correctly aligned characters. Errors come from edit alignment rather than comparing characters at identical indices. Insertions are absent from the denominator, so CER can exceed one. A short reference makes each error proportionally larger. CER measures transcription disagreement with the reference, not semantic severity.

69. Hugging Face Evaluate: Word Error Rate

    WER applies Levenshtein alignment to word sequences: WER=(S+D+I)/N, with reference words in the denominator. A one-character misspelling can count as one whole-word substitution, while different tokenization can change insertion and deletion counts. Applying this sequence metric to OCR is valid once the text and word-segmentation conventions are specified. Comparing WER values requires compatible references and tokenization rather than merely the same model name.

70. JiWER: Text Transformations and Alignment

    JiWER exposes word and character alignments and allows transformations of reference and hypothesis before scoring. Its example removes repeated spaces, strips boundaries, expands a contraction, and converts text into word lists. Therefore normalization is part of the metric contract: case folding, punctuation removal, whitespace handling, or equivalent-string substitutions can remove differences that raw scoring would count. Specify and apply the intended transformation consistently to both sides.

71. Introduction to Information Retrieval: Precision and recall

    Precision is TP/(TP+FP); recall is TP/(TP+FN). Applying these set definitions to extraction requires a reference set of field-value pairs, predicted pairs, and an explicit matching policy. With exact pair matching, a wrong value produces both an unmatched prediction (FP) and a missed reference pair (FN); missing pairs are FN and extra pairs FP. Abstaining removes predictions, not required reference pairs, so it can reduce coverage. Zero denominators require an explicit reporting convention.

72. Using Adaptive Turnaround Documents to Electronically Acquire Structured Data in Clinical Settings

    The CHICA pilot generated patient-specific paper forms, linked returned scans to records through barcodes and recognized entries within template-defined regions. Across six clinic days, 224 forms were printed and 221 unique scans received. Of those scans, 125 required no verification before data entry. The remaining forms averaged 1.7 fields needing correction; the authors reported verification taking approximately five to ten seconds. Digit recognition accuracy was 98.6%, illustrating that high character accuracy and whole-form readiness are different outcomes.

73. Amazon SageMaker AI: Human Loop Activation Conditions for Amazon Textract

    The documented review integration distinguishes key-value confidence from confidence in underlying word blocks. Review can be triggered by confidence conditions, failure to detect an important key or alias, or sampling regardless of confidence. Workers receive selected predictions, or a request to identify a missing key and value. This is a concrete example of targeted review plus auditing beyond low-score cases.

74. On Calibration of Modern Neural Networks

    Calibration asks whether predictions assigned confidence p are correct about proportion p of the time. Reliability diagrams compare observed accuracy with confidence in bins; expected calibration error averages absolute bin discrepancies weighted by bin population. Calibration differs from prediction accuracy. Selective risk instead measures errors among accepted predictions, and coverage is the accepted fraction. A threshold can alter risk and coverage without demonstrating calibrated probabilities.

75. Building an Agentic Platform

    Use judge feedback to drive another attempt before presenting an answer, rather than only attaching an uncertainty warning.

76. Selective Classification: Coverage and Conditional Risk

    A selective predictor combines prediction f with acceptance function g(x) in {0,1}; g=0 means abstention. Coverage is C=E[g(X)], the probability of accepting an input. Selective risk is R=E[loss(f(X),Y)g(X)]/C when C>0: expected loss conditional on acceptance. For N evaluated inputs and A accepted inputs, empirical coverage is A/N; with zero-one loss, empirical selective risk is errors among accepted inputs divided by A. Abstentions remain in the coverage denominator but leave the selective-risk denominator. Consequently lower reported risk can reflect a smaller selected population rather than an improved underlying predictor. Report coverage with risk and examine the risk-coverage curve across selection thresholds.

77. Failure Prediction in 2D Document Information Extraction with Calibrated Confidence Scores

    Basware's 2023 study trained an auxiliary confidence estimator to predict whether five important invoice fields were all correct: invoice number, issue date, subtotal, tax and total. It combined information across fields and calibrated its scores separately, without retraining the underlying extractor. On an independent 24,000-invoice test set, the authors estimated that automation coverage would increase from 65.6% to 73.2% at the production system's target precision. This evaluates document-level acceptance separately from individual field predictions.

78. OCRmyPDF: Advanced features — Debugging the intermediate files

    Retained intermediate files distinguish the rendered input page, image actually sent to OCR, preprocessing results and recognized text. For mixed pages, the OCR text artifact need not contain all page text.

79. Extracting Medical Information from Paper COVID-19 Assessment Forms

    Vanderbilt's document-review interface placed the scanned form with recognition overlays beside an editable form matching its layout; selecting a field highlighted the corresponding location in both views. The study analyzed 2,814 assessment forms and reported mean recognition accuracy of 70% against reviewers' results. Mean review time was 157 seconds per document. Removing the requirement to transcribe handwritten comments reduced reported annotation time from 188 to 156 seconds, showing that reduced work scope can change timing independently of recognition improvements.

80. Building AI Agents that actually automate Knowledge Work

    A document preprocessing layer should synchronize permissions and metadata alongside content, then parse and index it for agent access.

81. Building AI Agents that actually automate Knowledge Work

    Encode domain-specific parsing, extraction, and rule matching in an automation workflow that writes structured data to SQL and includes human review.

82. Docling: Chunking

    Docling describes a chunk as a portion of document content accompanied by metadata. Native chunking operates on the structured document rather than requiring a Markdown-first export. Its hierarchical representation carries headings and captions, and table chunks can repeat headers so separately consumed rows retain column context.

83. Structuring the Unstructured: Advanced Document Parsing for AI Workflows

    The PDF image extraction demo associates each source picture with its caption and embedded text elements.

84. AI Engineering with the Google Gemini 2.5 Model Family

    Pydantic schemas turn document extraction into typed data suitable for downstream APIs or databases, but selecting the right schema remains essential.

85. Introduction to Information Retrieval: Near-duplicates and shingling

    A k-shingle is a contiguous sequence of k terms; each document becomes a set of such sequences. Jaccard similarity is |A intersection B|/|A union B|. A threshold identifies substantial textual overlap. MinHash estimates this similarity because, under a random permutation, the probability that two sets share the minimum element equals their Jaccard similarity. Exact fingerprints and approximate overlap serve different purposes. Application inference: keep document identity, source location, revision identifier, content hash, validity and provenance separately from duplicate-cluster membership. A small change to a date, number or negation can be material despite high overlap.

86. Forget RAG Pipelines—Build Production-Ready AI Agents in 15 Minutes

    Carry hierarchical metadata and source bounding boxes into chunks rather than retaining only detached text.

87. SelectiveNet: A Deep Neural Network with an Integrated Reject Option

    Selective prediction pairs predictor f with selection function g: accept when g(x)=1, otherwise abstain. A confidence threshold or learned selection score determines acceptance. Coverage is phi = E[g(X)]; selective risk is E[loss(f(X),Y)g(X)]/phi for phi>0. On labeled evaluation data, measure coverage as accepted cases divided by all cases, and risk as average loss among accepted cases. Varying the threshold produces a risk–coverage curve. SelectiveNet jointly learns prediction and selection while optimizing risk subject to a target coverage constraint. This makes abstention measurable instead of treating a confidence statement as sufficient evidence of reliability.

88. Building AI Agents that actually automate Knowledge Work

    The Carlyle example places extraction and human review before an analyst copilot for insights and report generation.