Skip to content

By Careerflow Human Data Labs

ResumeBench: the frontier AI benchmark for resume extraction

18 agentic parsers and vLLMs evaluated on 300 curated resumes

Pages from the benchmark.

Head to head

Pick any two systems and see where they differ.

87.16

LlamaParse Extract Agentic Plus

A · Document API

+1.60A − B, points of F1

85.55

Reducto Deep Extract

B · Document API

A better on 138level on 60B better on 102

LlamaParse Extract Agentic Plus is ahead by 1.60 points on these 300 resumes.

Full comparison →
  • Many entries 83A 85.0 · B 84.3+0.7
  • Long bullet lists 101A 84.9 · B 84.5+0.5
  • Split across pages 85A 85.3 · B 84.1+1.2
  • Big skills list 60A 86.0 · B 85.8+0.3
  • Custom sections 105A 85.1 · B 82.6+2.5
  • Combined headings 37A 84.6 · B 82.7+1.9
  • Promotions 88A 87.4 · B 86.0+1.4
  • Partial dates 138A 87.0 · B 85.2+1.8
  • Overlapping jobs 69A 84.1 · B 85.9−1.8
  • Accented text 13A 92.0 · B 85.5+6.5

Accuracy, cost and speed

How accurate each system is, what it costs, how fast it runs.

  • Document API
  • Vision LLM
↖ Better: higher score, lower cost606570758081828384858687888990$0.50$1$2$5$10$20$50$100Cost per 1,000 pages (log scale) →F1 ↑ (stretched above 80)Claude Opus 4.8Gemini 3.5 FlashGPT-5.4 NanoGPT-5.5GPT-5.6ExtractMax ContextAgenticAgentic PlusCost-EffectiveDeep ExtractExtractQwen 3.5 9BQwen 3.5 35BQwen 3.8 27BQwen 3.8 FlashGemma 4 26BGLM 5.3 Flash

What’s in the benchmark

300 resumes across fields and formats. 20 are public with ground truth; the rest on request.

resumes
300
pages
837
entries
–
bullets
–
skills
–

Every value in the ground truth was checked against the page by hand.

Fields people work in

  • Engineering & trades4314%
  • Operations & programs3211%
  • Data & AI258%
  • Software development248%
  • Healthcare & science238%
  • Marketing, media & design238%
  • Finance & accounting217%
  • Law & public sector196%
  • Cloud, IT & security196%
  • Education & other196%
  • Tech leadership & product186%
  • Consulting, strategy & risk186%
  • Sales & customer service165%

Kinds of resume

  • Everyday resumes21271%
  • Expert-built (Canva / LaTeX)3211%
  • Scanned & rotated196%
  • Distractor layouts186%
  • Resume-builder templates103%
  • Handwritten93%

Page layout

  • Single column19565%
  • Two-column238%
  • Two-column sidebar5418%
  • Three-column207%
  • Table grid83%

Pages per resume

22
126
90
35
15
12
123456+

What recurs

  • Partial dates138
  • Custom sections105
  • Long bullet lists101
  • Promotions88
  • Split across pages85
  • Many entries83
  • Sidebar or grid82
  • Overlapping jobs69

What makes a resume hard

The resume features that cost parsers the most.

F1 points lost by the median system on resumes with each label. Which parser handles each kind best →

How a resume is scored

Same PDF, schema and instructions for every system. Each field and bullet is checked word for word. No judge model.

1Normalize

Tidy spaces, dashes and quotes. Case and punctuation still count.

Proficient in workflowProficient in workflow

2Align

Match each job to the same job in the truth, by company, title and years.

3Compare

Each field and bullet: 1 if it matches word for word, 0 if not.

4Score

F1 per section, then the average.

F1 0.667

A real job entry from the benchmark, as GPT-5.4 Nano read it

3 / 6values right

3 values differ, so this entry scores 3/6.

A resume’s score is the average of its sections. The board averages all 300 resumes.

A missed job lowers recall; an invented one lowers precision. Pick another system to see its answer.

The job “Brand Storyteller / Evangelist · TechVenture” on page 2 of the resume
  • CompanyTechVenture
  • Job titleBrand Storyteller / Evangelist
  • Bullet 1TechVenture is a developer-tools company (not a VC firm despite the name)truth: TechVenture is a developer-tools company (not a VC firm despite the name)
  • Bullet 2Produced 40+ technical blog posts, 6 conference talks, and 3 video tutorials reaching 250K+ devel- operstruth: Produced 40+ technical blog posts, 6 conference talks, and 3 video tutorials reaching 250K+ developers
  • Bullet 3Represented TechVenture at KubeCon NA 2018 and JSConf US 2019; booth traffic up 3× YoYtruth: Represented TechVenture at KubeCon NA 2018 and JSConf US 2019; booth traffic up 3× YoY
  • Bullet 4Ghost-wrote two bylined articles for the CEO published in TechCrunch and InfoQ
Every scoring rule 5 definitions
Precision
Correct ÷ returned. Drops when a system makes things up.
Recall
Correct ÷ expected. Drops when a system misses things.
Pairing
Jobs match on company, title and years; schools on institution, degree and years; skills one to one. Order doesn’t matter.
Empty values
Empty in both: ignored. Empty in one only: wrong.
Perfect resumes
Resumes with nothing wrong and nothing missing.
The schema 14 sections, 150+ fields
SectionCompared in the F1Also under Dates
basicsname, email, phone, city, state, country, LinkedIn, photo flag, target title–
experiencecompany, title, city / state / country, remote, role description, URL, every bulletdates, currently here
educationinstitution, field of study, degree, GPA, location, every bulletdates, currently studying
personalSummarythe summary text–
skillsskill names, de-duplicated, comma-split–
certificationsname, issuer, URLdate
customSectionstitle, summary, every bullet–
awardstitle, awarder, summary, URLdate
projectsname, URL, company, location, every bulletdates, in progress
volunteeringorganisation, role, location, summary, every bulletdates, currently volunteer
publicationstitle, publisher, summarydate
languages · interests · profilesname / network + URL–

Run it on your own parser

Harness, schema, prompt and scorer are open source. Grade your own pipeline or predictions.

careerflow/ResumeBench
$ pip install "resume-bench[runners] @ git+https://github.com/careerflow/ResumeBench"
$ hf auth login   # the dataset is gated: accept its terms on Hugging Face first
$ resume-bench download
$ resume-bench providers
$ resume-bench run llamaextract_agentic_plus --split test
$ resume-bench grade llamaextract_agentic_plus --split test
$ resume-bench grade-file my_predictions.jsonl --split test

Frequently asked questions (FAQs)

What is ResumeBench?

A public benchmark for resume extraction. 18 agentic parsers and vLLMs run on the same 300 resumes, and every field is checked against human-corrected ground truth, with cost and speed for each.

Who created ResumeBench?

Careerflow Human Data Labs, part of Careerflow.

What resumes are in ResumeBench?

138 anonymised real-world resumes, 9 handwritten and 153 synthetic (designed templates, scans, long careers and layouts built to trip parsers), 300 in all. The people in the published resumes are fictional.

Why did you create ResumeBench?

Parsers are sold on accuracy claims that are hard to check. We wanted one open test, on hard resumes, that scores every system the same way, so anyone can see what works and what it costs.

What is the best resume extraction model?

LlamaParse Extract Agentic Plus has the highest F1 (87.2). But the best one for you depends on what matters more: accuracy, cost or speed. See the full results table.

Which resume extraction model gives the best accuracy for the money?

Under $5 per 1,000 pages: Qwen 3.8 Flash (84.1 F1). Under $25 per 1,000 pages: LlamaParse Extract Agentic (85.5 F1). No budget limit: LlamaParse Extract Agentic Plus (87.2 F1, $54).

Can I download the resumes?

Yes. 20 are free on Hugging Face. For all 300, contact us.

Why are only some of the resumes public?

The full set and its human-corrected labels are licensed for research and commercial use under agreement. Keeping most of it held out also keeps the scores honest: no model can be trained or tuned on the test. The 20 published resumes show the method and let anyone check the scoring; the rest are shared for research and evaluation on request.

How often is ResumeBench updated?

New models are added after they come out, and the whole board is re-run at least once a quarter. See the changelog.