By Careerflow Human Data Labs
ResumeBench: the frontier AI benchmark for resume extraction
18 agentic parsers and vLLMs evaluated on 300 curated resumes
Pages from the benchmark.
Leaderboard
18 systems, 300 resumes, same inputs.
Each field checked word for word against the truth. Averaged per section, then per resume.
| 1 | LlamaParse Extract Agentic PlusDocument API | 87.16 | $54 | 70 s | 88.4 | 88.3 | 15.3 |
| 2 | Reducto Deep ExtractDocument API | 85.55 | ≈$40 | 65 s | 87.5 | 87.3 | 12.0 |
| 3 | LlamaParse Extract AgenticDocument API | 85.55 | $23 | 55 s | 86.8 | 87.3 | 13.7 |
| 4 | Claude Opus 4.8Vision LLM | 85.51 | $90 | 31 s | 86.8 | 87.5 | 10.3 |
| 5 | Gemini 3.5 FlashVision LLM | 85.45 | $35 | 43 s | 86.9 | 87.9 | 11.3 |
| 6 | GPT-5.6Vision LLM | 84.75 | $74 | 38 s | 85.9 | 87.2 | 12.0 |
| 7 | Extend Max ContextDocument API | 84.72 | ≈$50 | 43 s | 86.6 | 86.4 | 11.7 |
| 8 | Qwen 3.8 FlashVision LLM | 84.08 | $2.98 | 122 s | 85.4 | 86.2 | 8.4 |
| 9 | Qwen 3.8 27BVision LLM | 83.77 | $9.59 | 36 s | 85.4 | 85.5 | 10.3 |
| 10 | LlamaParse Extract Cost-EffectiveDocument API | 83.52 | $8.18 | 29 s | 85.1 | 85.3 | 8.7 |
| 11 | GPT-5.5Vision LLM | 83.50 | $87 | 48 s | 84.9 | 86.0 | 10.7 |
| 12 | Reducto ExtractDocument API | 81.70 | ≈$20 | 16 s | 83.7 | 84.0 | 5.3 |
| 13 | Extend ExtractDocument API | 80.97 | ≈$12 | 31 s | 82.9 | 83.2 | 5.3 |
| 14 | Qwen 3.5 35BVision LLM | 77.95 | $3.91 | 78 s | 78.8 | 83.7 | 1.3 |
| 15 | Qwen 3.5 9BVision LLM | 76.03 | $1.31 | 138 s | 78.9 | 80.3 | 2.0 |
| 16 | Gemma 4 26BVision LLM | 62.82 | $0.56 | 29 s | 69.1 | 66.2 | 1.3 |
| 17 | GPT-5.4 NanoVision LLM | 61.86 | $2.76 | 20 s | 70.1 | 66.3 | 0.3 |
| 18 | GLM 5.3 FlashVision LLM | 59.24 | $0.80 | 46 s | 71.8 | 62.0 | 7.7 |
Average over all 300 resumes. Cost per 1,000 pages, time per resume (median).
Accuracy, cost and speed
How accurate each system is, what it costs, how fast it runs.
- Document API
- Vision LLM
What’s in the benchmark
300 resumes across fields and formats. 20 are public with ground truth; the rest on request.
- resumes
- 300
- pages
- 837
- entries
- –
- bullets
- –
- skills
- –
Every value in the ground truth was checked against the page by hand.
Fields people work in
- Engineering & trades4314%
- Operations & programs3211%
- Data & AI258%
- Software development248%
- Healthcare & science238%
- Marketing, media & design238%
- Finance & accounting217%
- Law & public sector196%
- Cloud, IT & security196%
- Education & other196%
- Tech leadership & product186%
- Consulting, strategy & risk186%
- Sales & customer service165%
Kinds of resume
- Everyday resumes21271%
- Expert-built (Canva / LaTeX)3211%
- Scanned & rotated196%
- Distractor layouts186%
- Resume-builder templates103%
- Handwritten93%
Page layout
- Single column19565%
- Two-column238%
- Two-column sidebar5418%
- Three-column207%
- Table grid83%
Pages per resume
What recurs
- Partial dates138
- Custom sections105
- Long bullet lists101
- Promotions88
- Split across pages85
- Many entries83
- Sidebar or grid82
- Overlapping jobs69
What makes a resume hard
The resume features that cost parsers the most.
- Handwritten · 9 resumes−31.7
- Image only · 28 resumes−10.5
- Custom sections · 105 resumes−4.4
- Busy page · 18 resumes−4.1
- Many entries · 83 resumes−2.9
- Long bullet lists · 101 resumes−2.8
- Overlapping jobs · 69 resumes−2.5
- Split across pages · 85 resumes−2.2
F1 points lost by the median system on resumes with each label. Which parser handles each kind best →
Eight resumes up close
Open one for its pages, its ground truth and every system’s score.
How a resume is scored
Same PDF, schema and instructions for every system. Each field and bullet is checked word for word. No judge model.
1Normalize
Tidy spaces, dashes and quotes. Case and punctuation still count.
2Align
Match each job to the same job in the truth, by company, title and years.
3Compare
Each field and bullet: 1 if it matches word for word, 0 if not.
4Score
F1 per section, then the average.
A real job entry from the benchmark, as GPT-5.4 Nano read it
3 / 6values right
3 values differ, so this entry scores 3/6.
A resume’s score is the average of its sections. The board averages all 300 resumes.
A missed job lowers recall; an invented one lowers precision. Pick another system to see its answer.

- CompanyTechVenture
- Job titleBrand Storyteller / Evangelist
- Bullet 1TechVenture is a developer-tools company (not a VC firm despite the name)truth: TechVenture is a developer-tools company (not a VC firm despite the name)
- Bullet 2Produced 40+ technical blog posts, 6 conference talks, and 3 video tutorials reaching 250K+ devel- operstruth: Produced 40+ technical blog posts, 6 conference talks, and 3 video tutorials reaching 250K+ developers
- Bullet 3Represented TechVenture at KubeCon NA 2018 and JSConf US 2019; booth traffic up 3× YoYtruth: Represented TechVenture at KubeCon NA 2018 and JSConf US 2019; booth traffic up 3× YoY
- Bullet 4Ghost-wrote two bylined articles for the CEO published in TechCrunch and InfoQ
Every scoring rule 5 definitions
- Precision
- Correct ÷ returned. Drops when a system makes things up.
- Recall
- Correct ÷ expected. Drops when a system misses things.
- Pairing
- Jobs match on company, title and years; schools on institution, degree and years; skills one to one. Order doesn’t matter.
- Empty values
- Empty in both: ignored. Empty in one only: wrong.
- Perfect resumes
- Resumes with nothing wrong and nothing missing.
The schema 14 sections, 150+ fields
| Section | Compared in the F1 | Also under Dates |
|---|---|---|
| basics | name, email, phone, city, state, country, LinkedIn, photo flag, target title | – |
| experience | company, title, city / state / country, remote, role description, URL, every bullet | dates, currently here |
| education | institution, field of study, degree, GPA, location, every bullet | dates, currently studying |
| personalSummary | the summary text | – |
| skills | skill names, de-duplicated, comma-split | – |
| certifications | name, issuer, URL | date |
| customSections | title, summary, every bullet | – |
| awards | title, awarder, summary, URL | date |
| projects | name, URL, company, location, every bullet | dates, in progress |
| volunteering | organisation, role, location, summary, every bullet | dates, currently volunteer |
| publications | title, publisher, summary | date |
| languages · interests · profiles | name / network + URL | – |
Run it on your own parser
Harness, schema, prompt and scorer are open source. Grade your own pipeline or predictions.
careerflow/ResumeBench$ pip install "resume-bench[runners] @ git+https://github.com/careerflow/ResumeBench"
$ hf auth login # the dataset is gated: accept its terms on Hugging Face first
$ resume-bench download
$ resume-bench providers
$ resume-bench run llamaextract_agentic_plus --split test
$ resume-bench grade llamaextract_agentic_plus --split test
$ resume-bench grade-file my_predictions.jsonl --split test