Alibaba · Vision LLM
Qwen 3.5 9B on ResumeBench: 76.0 F1, rank 15 of 18
Qwen 3.5 9B ranks 15th of 18 on ResumeBench: 76.0 F1, $1.31 per 1,000 pages, 138 seconds per resume. It is 11.1 points behind LlamaParse Extract Agentic Plus.
8th of 11 vision LLMs. Strongest on table grid (−0.8 points), handwritten (−0.9 points) and accented text (−1.8 points). Weakest on scanned & rotated (−18.4 points), rotated & warped pages (−16.5 points) and hidden text & prompt injections (−14.4 points).
- Rank
- 15/18
- F1
- 76.0
- Precision
- 78.9
- Recall
- 80.3
- Perfect resumes
- 6/300
- Per 1,000 pages
- $1.31
- Median per resume
- 138 s
Averaged over 300 resumes. Dates (scored separately): 70.4. 90% finish within 490 s.
Where Qwen 3.5 9B wins and loses
F1 by kind of resume, against the median system.
Reading the page
| Kind of resume | # of resumes | F1 | Median | Gap | Rank |
|---|---|---|---|---|---|
| Handwritten | 9 | 50.6 | 51.5 | −0.9 | 12th |
| Image only | 28 | 61.2 | 71.4 | −10.1 | 14th |
| Sidebar or grid | 82 | 73.5 | 81.9 | −8.4 | 15th |
| Photo | 45 | 74.9 | 84.0 | −9.2 | 15th |
Distractors
| Kind of resume | # of resumes | F1 | Median | Gap | Rank |
|---|---|---|---|---|---|
| Rotated & warped pages | 21 | 66.5 | 82.9 | −16.5 | 14th |
| Hidden text & prompt injections | 14 | 70.2 | 84.5 | −14.4 | 14th |
| Content as images | 8 | 85.9 | 88.2 | −2.3 | 12th |
| Wording traps | 16 | 77.7 | 84.5 | −6.8 | 15th |
| Typos kept on purpose | 37 | 72.2 | 83.7 | −11.4 | 15th |
Extraction challenges
| Kind of resume | # of resumes | F1 | Median | Gap | Rank |
|---|---|---|---|---|---|
| Many entries | 83 | 73.9 | 81.9 | −8.0 | 15th |
| Long bullet lists | 101 | 73.2 | 81.4 | −8.2 | 15th |
| Split across pages | 85 | 73.4 | 82.2 | −8.8 | 15th |
| Big skills list | 60 | 76.3 | 83.6 | −7.3 | 15th |
| Custom sections | 105 | 73.0 | 80.8 | −7.8 | 15th |
| Combined headings | 37 | 72.2 | 82.7 | −10.5 | 15th |
| Promotions | 88 | 74.8 | 83.5 | −8.7 | 15th |
| Partial dates | 138 | 75.6 | 83.4 | −7.9 | 15th |
| Overlapping jobs | 69 | 72.2 | 81.7 | −9.6 | 15th |
| Accented text | 13 | 83.7 | 85.5 | −1.8 | 14th |
Noise
| Kind of resume | # of resumes | F1 | Median | Gap | Rank |
|---|---|---|---|---|---|
| Repeated header | 67 | 76.4 | 85.2 | −8.7 | 15th |
| References | 55 | 77.6 | 85.2 | −7.6 | 15th |
| Busy page | 18 | 67.6 | 80.1 | −12.5 | 15th |
Source
| Kind of resume | # of resumes | F1 | Median | Gap | Rank |
|---|---|---|---|---|---|
| Everyday resumes | 212 | 77.2 | 85.1 | −7.9 | 15th |
| Expert-built (Canva / LaTeX) | 32 | 79.1 | 83.6 | −4.5 | 14th |
| Resume-builder templates | 10 | 78.5 | 81.2 | −2.7 | 14th |
| Scanned & rotated | 19 | 64.5 | 82.9 | −18.4 | 14th |
| Distractor layouts | 18 | 80.4 | 85.3 | −5.0 | 15th |
Layout
| Kind of resume | # of resumes | F1 | Median | Gap | Rank |
|---|---|---|---|---|---|
| Single column | 195 | 76.7 | 84.0 | −7.2 | 15th |
| Two-column | 23 | 79.1 | 86.8 | −7.7 | 14th |
| Two-column sidebar | 54 | 69.7 | 80.0 | −10.3 | 15th |
| Three-column | 20 | 75.8 | 84.8 | −9.0 | 14th |
| Table grid | 8 | 93.3 | 94.1 | −0.8 | 11th |
Length
| Kind of resume | # of resumes | F1 | Median | Gap | Rank |
|---|---|---|---|---|---|
| Short | 148 | 78.1 | 87.1 | −9.0 | 16th |
| Medium | 90 | 78.0 | 84.7 | −6.7 | 15th |
| Long | 62 | 68.0 | 74.6 | −6.6 | 15th |
By section of the resume
F1 by resume section, against the median system.
| Section | Resumes | F1 | Median | Gap |
|---|---|---|---|---|
| Contact details | 299 | 92.6 | 93.5 | −1.0 |
| Experience | 299 | 77.2 | 86.3 | −9.1 |
| Education | 296 | 80.8 | 89.4 | −8.7 |
| Summary | 290 | 77.9 | 81.7 | −3.8 |
| Certifications | 246 | 74.7 | 81.1 | −6.4 |
| Awards | 127 | 55.2 | 75.4 | −20.2 |
| Skills | 293 | 83.2 | 92.1 | −8.9 |
| Languages | 172 | 80.0 | 84.3 | −4.3 |
| Custom sections | 214 | 36.4 | 49.9 | −13.5 |
| Interests | 68 | 84.3 | 91.7 | −7.4 |
| Volunteering | 80 | 80.2 | 84.5 | −4.3 |
| Profiles | 38 | 51.3 | 66.7 | −15.4 |
| Projects | 79 | 75.5 | 83.2 | −7.7 |
| Publications | 53 | 76.3 | 78.6 | −2.2 |
On the published resumes
Best and worst of the 20 public resumes.
| Resume | What makes it hard | F1 |
|---|---|---|
| Head of Events & Public Space Operations | Table grid with emoji bullets | 88.2 |
| Head of Digital Marketing | Canva design with a photo | 85.2 |
| Quality Systems Lead | Photo of a printout | 79.4 |
| Supervisory Human Resources Specialist, GS-0201-15 | Resume-builder template | 54.9 |
| senior Consultant - package Implementation | Handwritten resume | 48.7 |
| Senior ocularist and practice owner | Handwritten resume, six pages | 43.7 |
How it was run
Qwen3.5-9B (open weights), page images in, JSON schema out. Same PDF, schema and instructions as every other system, one run per resume (4 November 2026).
Run the open-source harness: list its pipelines, or grade your own predictions (GitHub):
$ pip install "resume-bench[runners] @ git+https://github.com/careerflow/ResumeBench"
$ hf auth login # the dataset is gated: accept its terms on Hugging Face first
$ resume-bench download
$ resume-bench providers
$ resume-bench grade-file my_predictions.jsonl --split testClosest alternatives
Nearest in F1.
| Model | F1 | Per 1,000 pages | Median time | Head to head |
|---|---|---|---|---|
| GPT-5.5 | 83.5 | $87 | 48 s | Qwen 3.5 9B vs GPT-5.5 |
| Reducto Extract | 81.7 | $20 | 16 s | Qwen 3.5 9B vs Reducto Extract |
| Extend Extract | 81.0 | $12 | 31 s | Qwen 3.5 9B vs Extend Extract |
| Qwen 3.5 35B | 77.9 | $3.91 | 78 s | Qwen 3.5 9B vs Qwen 3.5 35B |
Qwen 3.5 9B against every other model
- Qwen 3.5 9B vs LlamaParse Extract Agentic Plus
- Qwen 3.5 9B vs LlamaParse Extract Agentic
- Qwen 3.5 9B vs LlamaParse Extract Cost-Effective
- Qwen 3.5 9B vs Reducto Deep Extract
- Qwen 3.5 9B vs Extend Max Context
- Qwen 3.5 9B vs GPT-5.6
- Qwen 3.5 9B vs GPT-5.4 Nano
- Qwen 3.5 9B vs Claude Opus 4.8
- Qwen 3.5 9B vs Gemini 3.5 Flash
- Qwen 3.5 9B vs Gemma 4 26B
- Qwen 3.5 9B vs Qwen 3.8 Flash
- Qwen 3.5 9B vs Qwen 3.8 27B
- Qwen 3.5 9B vs GLM 5.3 Flash
Embed this result
Shows its current rank. Updates with each run.