DeepSeek-OCR Notes

Abstract

Original text

“compressing long contexts via optical 2D mapping”

“an optimal and manageable number of vision tokens”

“20x … accuracy still remains at about 60%”

Interpretation

Key point	Explanation
Problem	Long text creates many text tokens, which is expensive inside an LLM.
Method	Convert text into a 2D visual representation and encode it with fewer vision tokens.
Evidence	Under 10x compression, precision remains around 97%; at 20x compression it is still roughly 60%.

Key takeaway

This paper is not simply about using OCR to read a document. Rather, it treats OCR as a proxy task for studying vision-text compression.

It does not claim that vision tokens can fully replace text tokens. Instead, it explores the feasibility of compressing long contexts using the visual modality.

1. Introduction

Original text

“quadratic scaling with sequence length”

“visual modality as an efficient compression medium”

“A single image containing document text can represent rich information”

“reexamine vision-language models from an LLM-centric perspective”

“OCR tasks … bridging vision and language”

“Our work makes three primary contributions”

Interpretation

1. Problem: long text is too expensive

Processing long text inside an LLM is not only expensive because token count increases, but also because attention cost grows quickly with sequence length.

2. Core intuition: text can be represented visually

The same document may require a very long sequence of text tokens, but if it is rendered as a single document image, it may fit into a much smaller number of vision tokens.

This is the idea behind optical compression.

3. Reframing the view: rethinking VLMs

The authors do not treat VLMs merely as visual question answering models. Instead, they ask whether the image encoder can help LLMs process textual information more efficiently.

4. Why OCR is the right choice

OCR is an ideal proof-of-concept task because it naturally follows the pattern:

text image -> compressed visual representation -> reconstructed text

If reconstruction quality remains high, it means that visual tokens still preserve the textual information.

Three contributions

Contribution	Explanation
1. Compression-rate analysis	At around 9–10x compression, precision remains above 96%; at 10–12x, it is around 90%; at 20x, it falls to about 60%.
2. DeepEncoder	Uses window attention, global attention, and a 16x convolutional compressor to balance high resolution, low activation memory, and fewer vision tokens.
3. DeepSeek-OCR	Built from DeepEncoder + DeepSeek3B-MoE; on OmniDocBench it reaches strong end-to-end performance with very few vision tokens.

Practical note

In addition to OCR, the model also handles charts, chemical formulas, simple geometric figures, and natural images. The paper emphasizes that it is not limited to clean text pages.

The paper also mentions a production setting: 20 nodes, each with 8 A100-40G GPUs, producing 33 million pages of LLM/VLM training data per day.

Key takeaway

Long text can first be transformed into an image, then compressed to a small number of vision tokens. If OCR can still recover the text, that means the text information has actually been preserved under compression.

Long context is expensive -> represent text in the visual modality -> prove recoverability with OCR -> DeepEncoder makes it practical

Finally, the authors connect this idea back to long-context LLMs, agent systems, and optimization of VLM token allocation.

2. Related Works

2.1 Typical Vision Encoders in VLMs

Figure 2: Typical vision encoders in popular VLMs

Original text

“Current open-source VLMs employ three main types of vision encoders”

“all of which suffer from their respective deficiencies”

“Long vision tokens will slow down both prefill and generation”

Interpretation

Figure 2 surveys the three common types of vision encoders in open-source VLMs. All of them can process images, but in high-resolution document scenarios each has its own bottleneck.

1. Dual-tower: two visual branches

Aspect	Description
Representative models	Vary, early DeepSeek-VL
Approach	One branch handles low-resolution global context while another handles high-resolution local details; both are then fed into the LLM.
Strength	Parameters and activation memory are relatively manageable.
Problem	Requires two preprocessing pipelines, making deployment more complex and complicating encoder pipeline parallelism.
Limitation	High-resolution capabilities remain limited; complex PDFs or tiny text are not always handled well.

2. Tile-based: split large images into patches

Aspect	Description
Represented by	InternVL series, DeepSeek-VL2
Approach	A large image is split into many tiles, with a reduced global view added.
Strength	Can process very high-resolution images and reduces activation memory per encoder pass.
Problem	If the image is split too aggressively, structure across regions can be lost (tables, paragraphs, page layout).
Limitation	Native encoder resolution remains low; the global view is too small; vision token count can become large.

3. Adaptive resolution: dynamic resolution handling

Aspect	Description
Representative models	Qwen2-VL, Qwen2.5-VL, Qwen3-VL
Approach	Images are dynamically split based on original dimensions and processed at different resolutions.
Strength	Flexible and does not require fixed tile sizes.
Problem	Large images generate many vision tokens and high activation memory.
Limitation	Requires long training sequences; prefill and generation both become slower during inference.

Key takeaway

The common issue is the same tradeoff: to capture details, you end up with more vision tokens or higher activation memory; to reduce cost, you sacrifice resolution, global structure, or deployment simplicity.

Figure 2 sets up the motivation for DeepEncoder: DeepSeek-OCR needs an encoder that supports high resolution, low activation memory, few vision tokens, and easy deployment.

2.2 End-to-end OCR Models

Original text

“fundamentally transforming the traditional pipeline architecture”

“separate detection and recognition expert models”

“for a document containing 1000 words, how many vision tokens are at least needed for decoding?”

Interpretation

Traditional OCR usually follows a pipeline: detect text regions first, then recognize the text. End-to-end OCR instead integrates the steps into a single model, simplifying the image-to-text pipeline.

Type	Representative model	Main point
Academic paper OCR	Nougat	Shows that end-to-end models can convert a paper page directly into text or LaTeX without a traditional OCR pipeline.
OCR 2.0	GOT-OCR2.0	Not only reads text but also handles formulas, tables, charts, and geometric figures in structured images.
General VLM	Qwen-VL, InternVL	These are not specialized OCR models, but they strengthen document understanding and text recognition capabilities.

Additional note: synthetic image parsing

synthetic image refers to artificially generated or highly structured images such as formulas, tables, charts, geometric diagrams, page layouts, or screenshots.

parsing means more than reading text; it also means understanding the structure of the image, such as table rows and columns, LaTeX formulas, chart elements, or paragraph layout.

Therefore, synthetic image parsing is broader than ordinary OCR:

image -> markdown / LaTeX / table structure / layout elements / text content

Key transition

Most end-to-end OCR research focuses on recognition quality and task scope. This paper asks a more fundamental question:

For a document containing 1,000 words, how many vision tokens are minimally required for decoding?

This shifts OCR from a pure recognition task toward the study of how much text information can be preserved in compressed vision tokens.

3. Methodology

3.1 Architecture

Figure 3: DeepSeek-OCR architecture

Original text

“a unified end-to-end VLM architecture consisting of an encoder and a decoder”

“DeepEncoder is responsible for extracting image features and tokenizing as well as compressing visual representations”

Interpretation

DeepSeek-OCR consists of two major parts:

Image / document page
-> DeepEncoder (~380M: sam-base 80M + CLIP-large 300M with a 16x compressor)
-> compressed vision tokens
-> DeepSeek3B-MoE decoder (total 3B, activated ~570M during inference)
-> text / structured output

Module	Function
DeepEncoder	Reads the image, extracts features, and compresses vision tokens.
DeepSeek3B-MoE decoder	Generates OCR or parsing outputs from compressed vision tokens and the prompt.

Three building blocks inside DeepEncoder

Component	Role	Input -> output (example: 1024×1024 image)
sam-base (80M)	Perception. Uses window attention to process high-resolution details efficiently.	image -> 4096 patch tokens
16x compressor	Compresses tokens. Two conv layers reduce token count by 16x.	4096 -> 256
CLIP-large (300M)	Knowledge. Uses dense global attention on the smaller token set.	256 -> semantically rich vision tokens

One sentence summary: sam cheaply reads the image, the compressor reduces tokens, and CLIP carefully understands the remaining compact representation.

Key takeaway

The central idea is not merely replacing the decoder. The real contribution is designing a DeepEncoder that can compress high-resolution document pages into a small number of vision tokens.

3.2 DeepEncoder

Original text

“Capable of processing high resolutions”

“Low activation at high resolutions”

“Few vision tokens”

“Support for multiple resolution inputs”

“current open-source encoders cannot fully satisfy all these conditions”

Interpretation

This is the extension of Section 2.1: the three common encoders each have weaknesses, and none fully satisfies all the requirements needed by DeepSeek-OCR. Therefore, the authors design a new encoder called DeepEncoder.

DeepEncoder must satisfy five requirements:

Requirement	Meaning	Why it matters
High resolution	Read small text and fine document details clearly.	Without it, OCR accuracy suffers.
Low activation memory	Large images should not blow up GPU memory.	High-resolution processing often triggers OOM.
Few vision tokens	Do not pass excessive token cost to the LLM.	This is the core goal of the paper.
Multi-resolution support	Support different compression ratios.	Needed to study the tradeoff between compression rate and accuracy.
Reasonable parameter count	Keep deployment practical.	A model that is too large negates the cost benefit.

Why build a new encoder?

Compared with the designs in Section 2.1:

• Dual-tower: requires two preprocessing pipelines and has limited high-resolution capability.
• Tile-based: native resolution is too low and the image is split too aggressively, leading to fragmented structure.
• Adaptive resolution: large images produce many vision tokens and high activation memory.

Therefore, the authors combine ideas into a new design.

3.2.1 Architecture of DeepEncoder

Original text

“window attention”

“dense global attention”

“16x downsampling of vision tokens”

“we remove the first patch embedding layer since its input is no longer images but output tokens from the previous pipeline”

Interpretation

DeepEncoder is built from a two-stage backbone with a compression module in the middle:

High-resolution image
-> [Perception] sam-base 80M: window attention for local detail
-> [Compress] 16x convolutional compressor: reduce tokens by 1/16
-> [Knowledge] CLIP-large 300M: global attention for holistic understanding
-> vision tokens

Design intuition: each stage does a different job

DeepEncoder splits image understanding into three stages, each at a different level of abstraction:

Stage	What it does	One-line summary
sam-base	Splits the high-resolution image into 4096 tokens, with each token capturing local details like edges, textures, and strokes.	It cheaply scans the entire image.
16x compressor	Merges 4096 tokens into 256 more compact tokens spatially.	It removes redundancy.
CLIP-large	Lets the 256 tokens attend to each other, enriching each token with local detail + global context.	It carefully reads the remaining essence.

Three common misconceptions

• sam is not just a patchifier: patching is only the entry step; its real role is to turn each patch into a meaningful token feature using window attention. This operation does not require a huge parameter count.
• After sam, the image is not gone: the compressor and CLIP operate in token space, not pixel space. The intuitive notion of “shrinking the image” should be replaced by “merging neighboring tokens.”
• CLIP is not simply turning pictures into semantic tokens: it sees 256 tokens that already exist. Its job is to allow these tokens to attend globally and upgrade each token from “local detail only” to “local detail + global context.” The number of tokens stays the same; their information becomes denser.

Window vs. global attention (source of requirement 2)

• Global attention: every token attends to every other token; this creates an $N \times N$ relation map. When tokens become too many, memory usage explodes.
• Window attention: each token only attends within a local window; the relation map is much smaller, which keeps memory manageable.

The tradeoff is that long-range information is not visible to every token, but this is acceptable for sam because its role is local detail capture (edges, textures, strokes). Global understanding happens later in CLIP.

The memory-saving requirement is achieved mainly by this design.

Core tradeoff

Window attention: cheap, but local only.
Global attention: powerful, but O(n²).
DeepEncoder strategy = place the expensive part (global attention) only on the already compressed small token set.

In other words: sam cheaply scans high-resolution details, the compressor removes redundancy, and CLIP performs expensive global reasoning on the few remaining tokens.

Detail 1: sam uses patch size 16

For a 1024 × 1024 image:

$$ \frac{1024}{16} \times \frac{1024}{16} = 4096 $$

This produces 4096 patch tokens at the start.

Although 4096 sounds large, sam only has 80M parameters and uses window attention rather than full-image pairwise attention, so activation memory remains manageable.

Detail 2: the 16x compressor is a 2-layer conv module

The compression module is inspired by Vary and uses two convolutional layers:

Parameter	Value
Kernel size	3
Stride	2
Padding	1
Channel	256 -> 1024

Each conv layer with stride 2 performs 2D downsampling by 4x; combining two layers gives 4 × 4 = 16x.

$$ \frac{4096}{16} = 256 $$

Thus, before global attention, only 256 vision tokens remain.

Detail 3: CLIP removes its first patch embedding layer

Original CLIP first applies a patch embedding layer to convert pixels into patches. In DeepEncoder, the input is no longer an image but the compressed token sequence from the previous stage. Therefore, this layer is removed.

This is a key point: DeepEncoder is not just sam + CLIP concatenated; the backbone is restructured and adapted as a token-based pipeline.

Key takeaway

DeepEncoder can be summarized as:

Use cheap window attention to preserve high-resolution detail, then use expensive global attention on a much smaller token set.

This is the mechanism that allows simultaneous high resolution, few tokens, and low activation memory.

3.2.2 Multiple Resolution Support

Figure 4: Multiple resolution modes

Table 1: Multi resolution support of DeepEncoder

Original text

“test how many vision tokens are needed for decoding”

“support a variable number of vision tokens”

“native resolution and dynamic resolution”

“dynamic interpolation of positional encodings”

“Tiling is a form of secondary window attention”

Interpretation

Why multi-resolution support is necessary

• Research motivation: To plot a compression-rate vs. accuracy curve (Table 2), the model must support multiple compression settings.
• Application motivation: Different documents require different token budgets. A slide may need only 64 tokens, while a newspaper may need thousands.

How multi-resolution support works

Two key techniques are used:

1. Dynamic interpolation of positional encodings: positional embeddings are not fixed; when the input resolution changes, the model interpolates the correct positional layout so the same model can accept different image sizes.
2. Simultaneous multi-mode training: the model is trained with many resolution modes together so a single DeepSeek-OCR model can operate in different settings.

Two major resolution types

Type	Mode	Use case
Native resolution	Tiny / Small / Base / Large	Fixed resolutions for testing different compression rates.
Dynamic resolution	Gundam / Gundam-M	High-resolution document handling, such as newspapers or very large pages.

Native resolution token counts:

Mode	Resolution	Tokens	Process
Tiny	512	64	Resize
Small	640	100	Resize
Base	1024	256	Padding
Large	1280	400	Padding

Why Tiny/Small use resize while Base/Large use padding

This is not simply a choice between “shrink” and “pad.” It is a strategy for handling different aspect ratios.

• Resize (Tiny/Small): the image is resized to a fixed height/width regardless of the original ratio, which can distort the page.
• Padding (Base/Large): first resize to fit the aspect ratio, then pad the remaining area to preserve the original layout.

Strategy	Aspect ratio	Token efficiency
Resize	Distorted	Uses content densely
Padding	Preserved	Wastes some tokens on empty regions

Why are small modes resize-based and large modes padding-based? It is a token-budget tradeoff.

• Tiny/Small have only 64/100 tokens, so wasting a few padded regions can cost 5–10% of the budget.
• Base/Large have more tokens (256/400), so preserving aspect ratio is more valuable than squeezing every token.

In short: the scarcer the token budget, the less room for wasted padding.

The number of valid vision tokens after padding is smaller than the total tokens produced:

$$ N_{\text{valid}} = \left\lceil N_{\text{actual}} \times \left[ 1 - \frac{\max(w,h)-\min(w,h)}{\max(w,h)} \right] \right\rceil $$

In plain language: tokens corresponding to empty padding regions are removed from the actual usable count.

Gundam mode: local tiles + global view

First ask the question: a newspaper page may be 3000×4000 pixels. Feeding it directly into the Base mode would shrink it to 1024 and blur the text, making it unreadable. Therefore, high-resolution documents cannot be processed as a single whole image.

Gundam’s strategy uses two parallel views:

• Local tiles: crop the image into multiple 640×640 high-resolution patches, each passed through the Small mode (100 tokens × number of tiles).
• Global view: resize the whole page to 1024×1024 and pass it through the Base mode (256 tokens).

Combined:

$$ N_{\text{vision}} = n \times 100 + 256 $$

This is similar to how human vision works:

• tiles are like foveal vision: high-resolution reading of local details.
• the global view is like peripheral vision: low-resolution layout understanding.

Without tiles, the model sees the layout but loses detail. Without the global view, the model sees local details but loses page context. Together, it has both.

Why use Small mode for tiles instead of Base mode? Because tiles are multiplied by count. A small tile is much cheaper. For example, 8 Small tiles = 800 tokens; 8 Base tiles = 2048 tokens. The global view is a single image, so it can afford a larger budget.

Why does a small number of tiles (2–9) suffice? This is one of the key reasons Gundam avoids the weaknesses of tile-based methods. DeepEncoder’s native resolution is already large enough that a page can be covered with a modest number of tiles, instead of dozens of fragmented patches. This helps preserve document structure across regions.

The paper makes an elegant framing:

Tiling is a form of secondary window attention.

This means tiling is a second-order local processing mechanism that further reduces activation pressure while keeping local details.

Edge cases

• Small-image fallback: if the image is smaller than 640 in both dimensions, n = 0 and the model falls back to Base mode.
• Gundam-Master: a more expensive variant with 1024 local + 1280 global (tile tokens increase from 100 to 256, and global view from 256 to 400). This is not part of the main training runs; it is a continued training variant intended to improve quality after the model is already working well.

Key takeaway

DeepEncoder is not merely a compressor; it is a multi-resolution encoder.

Compression rate is determined by the mode chosen, e.g. text tokens / vision tokens. A document with 1,000 text tokens may correspond to:

• Tiny -> 15x compression
• Small -> 10x
• Base -> 4x
• Large -> 2.5x

This directly addresses the five requirements introduced earlier:

Requirement	What would happen without it
High resolution	Cannot support Base/Large modes.
Low activation memory	High-res inputs would OOM.
Few vision tokens	Cannot reach Tiny/Small low-token settings.
Multi-resolution	Cannot sweep different compression levels.
Reasonable params	The model becomes too heavy to train and deploy.

All five are necessary to produce the compression-rate vs. accuracy curve in Table 2.

3.3 The MoE Decoder

Original text

“activates 6 out of 64 routed experts and 2 shared experts”

“570M activated parameters”

“reconstructs the original text representation from the compressed latent vision tokens”

“very suitable for domain-centric (OCR for us) VLM research”

Interpretation

The decoder is DeepSeek-3B-MoE:

Item	Value
Total params	Approx. 3B
Activated experts	6 routed + 2 shared
Activated parameters	Approx. 570M

Why MoE instead of dense models

The paper argues that MoE is well suited for this task because:

• Expressive power: It retains the capacity of a 3B model while processing domain-specific tasks like OCR.
• Inference cost: It activates only around 570M parameters, making it close to a smaller dense model in runtime efficiency.

This is a sweet spot for domain-centric VLM research such as OCR.

Decoder objective

The decoder maps compressed vision tokens back to decoded text tokens:

$$ f_{\text{dec}}: \mathbb{R}^{n \times d_{\text{latent}}} \rightarrow \mathbb{R}^{N \times d_{\text{text}}} $$

$$ \hat{X} = f_{\text{dec}}(Z), \quad n \leq N $$

In plain language: the model reconstructs a longer text representation from a smaller number of compressed vision tokens.

n < N is the condition under which compression is truly meaningful: a few vision tokens can generate more text tokens.

Key takeaway

This is not just OCR; it is a test of whether a small number of vision tokens is enough to reconstruct a larger text representation.

It also suggests a broader conjecture: if the LLM is exposed to optical pretraining, this capability may emerge naturally without hard-coded intervention.

3.4 Data Engine

Original text

“OCR 1.0 data”

“OCR 2.0 data”

“General vision data”

“Text-only data”

Interpretation

DeepSeek-OCR is trained not only on traditional OCR data, but also on structured visual data, general vision tasks, and text-only data.

Data type	Purpose	Content
OCR 1.0	Traditional OCR ability	Document OCR, natural-scene OCR
OCR 2.0	Complex parsing	Charts, chemical formulas, geometry
General vision	Preserve visual interface	Captioning, detection, grounding
Text-only	Preserve language ability	8192-token text-only pretraining data

Overall composition:

OCR data 70%
General vision data 20%
Text-only data 10%

3.4.1 OCR 1.0 Data

Figure 5: OCR 1.0 fine annotations

OCR 1.0 includes two categories: document OCR and natural-scene OCR.

Document OCR

• Source: PDFs on the web.
• Scale: around 30M pages covering roughly 100 languages.
  • Chinese and English account for around 25M.
  • Other languages account for about 5M.

Two types of ground truth are used:

Annotation type	Use
Coarse annotations	Extract text directly from PDFs using fitz; good for scale, but lacking layout structure. Mainly used for teaching the model to recognize characters, especially in smaller languages.
Fine annotations	Use layout models such as PP-DocLayout plus OCR tools such as MinerU or GOT-OCR2.0 to mark layout, coordinates, and text. Much smaller in scale, but structurally rich. Chinese and English each have about 2M pages.

Figure 5 shows fine annotations: each text segment is annotated with position and category labels, with coordinates normalized to 1000 bins.

Different prompts are used for the different annotation types.

How the model flywheel handles low-resource languages

For languages without mature OCR models:

1. Use fitz to split PDFs into patches with corresponding text.
2. Train a GOT-OCR2.0 model on that data.
3. Use the new model to annotate more patches.
4. Accumulate a dataset of around 600K samples.

This is a classic model flywheel approach.

Additional data

• 3M Word documents: mainly improve formulas and HTML tables, without layout annotations.
• Natural-scene OCR: from LAION + Wukong, labeled with PaddleOCR, with 10M Chinese and English samples each.

3.4.2 OCR 2.0 Data

Figure 6: OCR 2.0 parsing data

OCR 2.0 is not just text recognition; it focuses on parsing more complex structured images.

Type	Source	Scale	Output format
Chart	pyecharts + matplotlib rendering (line/bar/pie/composite)	10M	HTML table
Chemical formula	PubChem SMILES + RDKit rendering	5M	SMILES
Plane geometry	Slow Perception with translation-invariant augmentation	1M	Dictionary with line segments, endpoints, and line types

A few implementation details:

• Charts use HTML tables rather than a dictionary-based format because it is more token-efficient.
• Plane geometry models each line segment using a perception-ruler size = 4 and follows the Slow Perception approach.
• Translation-invariant augmentation ensures that the same geometric figure at different positions is treated as the same target, increasing data diversity.

3.4.3 General Vision Data

• About 20% of total data.
• Sourced from DeepSeek-VL2 captioning, detection, and grounding tasks.
• DeepSeek-OCR is not a general VLM, but this data helps preserve a general visual interface and supports future extension research.

3.4.4 Text-only Data

• About 10% of data, all processed to 8192 tokens.
• Purpose: maintain the decoder’s language ability and avoid overfitting purely to image-to-text generation.

Key takeaway

The purpose of the data engine is not to maximize raw volume but to balance four abilities:

understand structured documents (OCR 1.0 + 2.0) + preserve general vision + preserve language ability

3.5 Training Pipelines

Original text

“Training DeepEncoder independently”

“Training the DeepSeek-OCR”

Interpretation

Training happens in two stages:

Step 1: Train DeepEncoder independently.
Step 2: Train the full DeepSeek-OCR model.

3.5.1 Stage 1: independent DeepEncoder training

Setting	Value
Framework	Inspired by Vary; uses a compact LM for next-token prediction
Data	All OCR 1.0 + 2.0 + 100M general data sampled from LAION
Epochs	2
Batch size	1280
Optimizer	AdamW + cosine annealing
Learning rate	5e-5
Sequence length	4096

The goal is to let DeepEncoder learn how to transform images into decodable vision tokens before being integrated into the full pipeline.

3.5.2 Stage 2: full DeepSeek-OCR training

Platform: HAI-LLM.

Pipeline parallel (PP) is split into 4 stages:

Stage	Contents	Training status
PP0	sam + 16x compressor	Frozen; used as a visual tokenizer
PP1	CLIP	Trainable; treated as the input embedding layer
PP2	first 6 decoder layers	Trained
PP3	last 6 decoder layers	Trained

Training settings:

Setting	Value
Hardware	20 nodes × 8 A100-40G (160 GPUs total)
Data parallel	40
Global batch size	640
Optimizer	AdamW + step-based scheduler
Learning rate	3e-5
Speed (text-only)	90B tokens/day
Speed (multimodal)	70B tokens/day

Key design decisions

• Freeze sam + compressor: treat them as a source of vision tokenization and do not update their weights. This matches the idea that DeepEncoder behaves like a visual tokenizer.
• Do not freeze CLIP: treat CLIP as the input embedding layer and train it jointly with the decoder.
• Split the 12 decoder layers into two segments: this is simply a load-balancing decision for pipeline parallelism.

Key takeaway

The methodology is centered on DeepEncoder: first preserve high-resolution details using local attention, then reduce token count with a 16x compressor, then recover text or structured output with the decoder.

The training scheme uses a two-stage pipeline plus parallel training to keep the visual tokenizer frozen and focus the main optimization on the language model.

4. Evaluation

4.1 Vision-text Compression Study

Table 2: Vision-text compression ratio

Original text

“we use … Fox benchmarks”

“approximately 97% … 10x compression”

“nearly 10x lossless contexts compression”

“20x … precision can still approach 60%”

Interpretation

This section is the core of the paper: how much text can be reconstructed from a small number of vision tokens?

Experimental setup

Item	Value
Benchmark	Fox (English document portion)
Tokenizer	DeepSeek-OCR tokenizer (vocab ≈ 129k)
Filter range	Documents with 600–1300 text tokens, total 100 pages
Test modes	Tiny (64 tokens, 512), Small (100 tokens, 640)
Prompt	<image>\nFree OCR.

Note: the model output format is not identical to the Fox format, so the measured performance is slightly better than the raw table numbers.

Table 2: summary of results

Read the table as follows: the x-axis is the text-token range; the y-axis compares precision and compression under the 64-token and 100-token vision settings.

Range	Observation
Within 10x compression	Precision is about 97%, nearly lossless.
Around 10x–12x	Precision drops to about 90%, still decent but noticeable.
Near 20x	Precision falls to about 60%.

Why performance drops as compression increases

The authors suggest two likely reasons:

1. Longer documents become more layout-complex: page structure becomes harder to reconstruct.
2. Low-resolution rendering makes long text blur: at 512 or 640 resolution, too many characters are packed into a tiny visual space.

A useful interpretation is:

• The first problem can be mitigated by rendering text onto a single page layout.
• The second can be seen as a kind of forgetting mechanism: old information naturally becomes fuzzy as it is compressed.

This idea is developed more fully in Section 5.

Key takeaway

• Around 10x compression is nearly lossless -> good for historical context compression in conversations.
• At 20x, precision remains around 60% -> this is not hopeless; it is suitable for gradual forgetting.
• The extra cost is minimal: a VLM already needs a vision encoder, so integrating OCR is a natural extension.

4.2 OCR Practical Performance

Table 3: OmniDocBench results

Table 4: Per-category edit distances

How to read these tables: all numbers are edit distance, where smaller is better. 0 means perfectly correct, 1 means fully wrong, and 0.1 means roughly 10% of characters need correction. In Table 3, the columns text / formula / table / order correspond to edit distances for those sub-tasks.

Original text

“Requiring only 100 vision tokens … surpasses GOT-OCR2.0 which uses 256 tokens”

“with 400 tokens … on-par performance with state-of-the-art”

“Using fewer than 800 tokens (Gundam) … outperforms MinerU2.0 which needs nearly 7,000 vision tokens”

Interpretation

This section shows that DeepSeek-OCR is not only a research proof-of-concept; it is also practically competitive.

Key comparisons on OmniDocBench (Table 3)

Model	Vision tokens	Result
GOT-OCR2.0	256	DeepSeek-OCR with 100 tokens outperforms it.
State-of-the-art	-	DeepSeek-OCR with 400 tokens (valid ~285) reaches comparable results.
MinerU2.0	~7000	DeepSeek-OCR in Gundam mode (<800 tokens) outperforms it.

In other words, on the same benchmark, DeepSeek-OCR achieves similar or better results using far fewer vision tokens.

Per-document-type breakdown (Table 4)

Different document types demand different token budgets:

Type	Minimum suitable mode	Why
Slides	Tiny (64)	Small page size and low character count.
Books, reports	Small (100)	Many pages remain under around 1,000 text tokens, so compression stays under 10x.
Newspapers	Gundam / Gundam-M	A newspaper page may contain 4000–5000 text tokens, which exceeds the range where simple 10x compression is enough.

This reinforces the observation from Section 4.1: as long as the document remains in the range where 10x compression is acceptable, performance stays strong.

Key takeaway

• DeepSeek-OCR reaches near-top-tier performance on OmniDocBench while using extremely few vision tokens.
• Different document types need different token budgets, and the model supports that dynamically.
• This makes contexts optical compression practically viable, especially as a generator of large-scale LLM/VLM pretraining data.

4.3 Qualitative Study

The paper uses three subsections to show that the model does more than standard OCR:

Subsection	Capability	Corresponding figure
4.3.1 Deep parsing	Charts, geometry, formulas, natural images	Figure 7–10
4.3.2 Multilingual recognition	Around 100 languages, with layout and non-layout outputs	Figure 11
4.3.3 General vision understanding	Captioning, detection, grounding	Figure 12

4.3.1 Deep Parsing

Figure 7: Deep parsing on financial chart

Figure 8: Deep parsing on book with natural image

Figure 9: Deep parsing on chemistry document

Figure 10: Deep parsing on plane geometry

Key concept: secondary model call.

The model first converts a document into Markdown, and if it encounters charts or other structured elements, it calls itself again to generate a structured result. The whole process can be driven by the same prompt and does not require an external pipeline.

Examples:

• Financial chart -> HTML table
• Book with natural image -> dense caption
• Chemistry document -> SMILES
• Simple geometry -> dictionary format with lines and coordinates

4.3.2 Multilingual Recognition

Figure 11: Multilingual OCR (Arabic, Sinhala)

The model supports around 100 languages in PDF documents. The prompt can be one of:

• <image>\nFree OCR. for plain text output.
• <image>\n<|grounding|>Convert the document to markdown. for layout-aware structured output.

Figure 11 demonstrates Arabic and Sinhala examples.

4.3.3 General Vision Understanding

Figure 12: General vision capabilities

Because about 20% of the training data comes from general vision tasks, the model also retains capabilities such as:

• image description
• object detection
• grounding

Note: because it is not fine-tuned as a chatbot, it is not a general chat model. Some capabilities only appear when using completion-style prompts.

Key takeaway

DeepSeek-OCR is not just an OCR tool; it is a structured data generator built around OCR. The secondary-call design is what elevates it from a recognition engine to a structured information extractor.

5. Discussion

Figure 13: Forgetting mechanism analogy

Original text

“implementing optical processing for dialogue histories beyond k rounds”

“render previous rounds of historical text onto images for initial compression”

“progressively resizing older images to achieve multi-level compression”

“human memory decay over time and visual perception degradation over spatial distance”

“theoretically unlimited context architectures”

Interpretation

This section is the most conceptual and exciting part of the paper. Sections 4.1–4.3 are about OCR, but Section 5 steps back and argues that OCR is only a means to a larger goal: building a better long-context architecture for LLMs.

Step 1: the conservative application — compressing multi-turn dialogue history

After more than k conversation rounds, older turns are rendered as images.
Then DeepEncoder compresses them into a few vision tokens.
These tokens are passed back to the LLM as context.

Even with this simple application, 10x compression is near-lossless (as shown in Section 4.1), which means we can reduce chat history cost by roughly an order of magnitude.

This is already an engineering-useful idea.

Step 2: the more ambitious version — forgetting mechanism

Figure 13 frames a conceptual analogy:

Human memory: recent events -> clearer, old events -> blurrier.
Visual perception: close objects -> detailed, far objects -> blurrier.
DeepSeek-OCR: Large -> Base -> Small -> Tiny

All three systems share the same idea: information is not deleted abruptly; it is gradually degraded in resolution.

A concrete implementation could be:

Recent dialogue -> Large mode (high detail, high cost)
Older dialogue -> Base / Small mode (medium detail, lower cost)
Older still -> Tiny mode (very blurry, minimal cost)
Very old -> drop it

The total token budget stays constant: as new context arrives, older context is downsampled or degraded, so the system never exceeds its budget.

Step 3: why this architecture is attractive

• Theoretically unlimited context: because the total token budget remains bounded, the system does not blow up.
• Biologically plausible: it matches how human memory works—recent memories are vivid, while older ones become fuzzy.
• No extra cost: vision encoders already exist in VLMs, so this idea leverages infrastructure that is already there.

This creates a path toward theoretically unlimited context + roughly constant cost + biologically plausible memory behavior—something that most long-context methods do not achieve simultaneously.

Step 4: the authors’ honesty

The final part of the discussion acknowledges that this is early-stage work:

“This is early-stage work that requires further investigation.”

Open issues include:

• When should information be degraded? (rolling window, event-based trigger, attention-based trigger?)
• Can the LLM attend correctly to degraded vision tokens?
• Should different content types (code, tables, natural language) follow different forgetting curves?

These are future challenges, but the paper already demonstrates an important milestone: small vision tokens can preserve a large amount of textual information.

6. Conclusion

Original text

“DeepSeek-OCR … preliminarily validate the feasibility of contexts optical compression”

“decode text tokens exceeding 10 times the quantity from a small number of vision tokens”

“digital-optical text interleaved pretraining, needle-in-a-haystack testing”

Interpretation

The conclusion summarizes the paper on two levels:

1. Research level: DeepSeek-OCR provides preliminary validation that optical context compression is feasible. A modest number of vision tokens can reconstruct more than 10x as many text tokens.
2. Application level: DeepSeek-OCR itself is a practical model for producing large-scale LLM/VLM training data.

The authors also honestly point to future work:

• Digital-optical text interleaved pretraining: combine plain text and rendered text images in pretraining to test whether optical compression can become an intrinsic capability of LLMs.
• Needle-in-a-haystack style evaluation: evaluate whether compressed information remains retrievable in long-context settings.

Key takeaway

The story of the paper is:

Long context is expensive
-> use the visual modality as a compression medium
-> OCR is the most direct measurable task
-> DeepEncoder solves the tradeoff between high resolution, few tokens, and low activation memory
-> 10x compression is almost lossless, 20x still retains about 60%
-> this can extend into forgetting-style long-context architectures

OCR is the means, not the final goal. The true goal is to discover a cheaper path to long-context processing inside LLMs.