Open beta Zabio is in open beta: anyone can sign up and use it in production. Models, prices and features can still change while we learn from the first customers. Tell us what breaks at hello@zabio.ai.
Benchmarks / Models / DeepSeek / deepseek-flash Model from DeepSeek
deepseek-flash benchmarks On Math (hardest), thinking high gets 98.2% right (range 92.3 to 99.6) against 73.3% with thinking off (range 62.6 to 81.9), for 2.9 times the cost.
On SQL and data queries (hardest), thinking low gets 100.0% right (range 95.3 to 100.0) against 88.5% with thinking off (range 79.5 to 93.8), for 5.7 times the cost.
Model id deepseek-flash
Price per 1M tokens $0.30 in, $1.20 out
Context window 1,048,576
Processing location DeepSeek, China
Zabio default Thinking off
Tested 25 September 2026 Math Right answers by difficulty and thinking setting Each dot is one thinking setting, and its bar is the 90% range.
Thinking off Thinking low Thinking high50 60 70 80 90 100 Easy Medium Hard Hardest Easy, Thinking off: 100.0% (90.2 to 100.0) Easy, Thinking low: 100.0% (90.2 to 100.0) Easy, Thinking high: 100.0% (90.2 to 100.0) Medium, Thinking off: 94.5% (87.1 to 97.8) Medium, Thinking low: 98.2% (92.3 to 99.6) Medium, Thinking high: 96.4% (89.6 to 98.8) Hard, Thinking off: 96.3% (89.4 to 98.8) Hard, Thinking low: 100.0% (95.2 to 100.0) Hard, Thinking high: 100.0% (95.2 to 100.0) Hardest, Thinking off: 73.3% (62.6 to 81.9) Hardest, Thinking low: 96.4% (89.6 to 98.8) Hardest, Thinking high: 98.2% (92.3 to 99.6) Cost per 1,000 tasks At list price, by difficulty. Thinking costs more because the model writes more tokens before it answers.
$0.2 $1 $5 Easy Easy, Off: $0.36 Off $0.36 Easy, Low: $0.27 Low $0.27 Easy, High: $0.32 High $0.32 Medium Medium, Off: $0.59 Off $0.59 Medium, Low: $0.62 Low $0.62 Medium, High: $1.04 High $1.04 Hard Hard, Off: $1.66 Off $1.66 Hard, Low: $2.26 Low $2.26 Hard, High: $3.00 High $3.00 Hardest Hardest, Off: $3.16 Off $3.16 Hardest, Low: $6.38 Low $6.38 Hardest, High: $9.18 High $9.18 The numbers behind this chart are in the table below it.
SQL and data queries Right answers by difficulty and thinking setting Each dot is one thinking setting, and its bar is the 90% range.
Thinking off Thinking low Thinking high50 60 70 80 90 100 Easy Medium Hard Hardest Easy, Thinking off: 100.0% (90.2 to 100.0) Easy, Thinking low: 100.0% (90.2 to 100.0) Easy, Thinking high: 100.0% (90.2 to 100.0) Medium, Thinking off: 96.4% (89.6 to 98.8) Medium, Thinking low: 96.4% (89.6 to 98.8) Medium, Thinking high: 98.2% (92.3 to 99.6) Hard, Thinking off: 93.9% (86.3 to 97.4) Hard, Thinking low: 100.0% (95.3 to 100.0) Hard, Thinking high: 100.0% (95.3 to 100.0) Hardest, Thinking off: 88.5% (79.5 to 93.8) Hardest, Thinking low: 100.0% (95.3 to 100.0) Hardest, Thinking high: 100.0% (95.3 to 100.0) Cost per 1,000 tasks At list price, by difficulty. Thinking costs more because the model writes more tokens before it answers.
$0.2 $1 $5 Easy Easy, Off: $0.24 Off $0.24 Easy, Low: $0.35 Low $0.35 Easy, High: $0.42 High $0.42 Medium Medium, Off: $0.45 Off $0.45 Medium, Low: $1.26 Low $1.26 Medium, High: $1.87 High $1.87 Hard Hard, Off: $0.58 Off $0.58 Hard, Low: $2.10 Low $2.10 Hard, High: $3.48 High $3.48 Hardest Hardest, Off: $0.75 Off $0.75 Hardest, Low: $4.29 Low $4.29 Hardest, High: $7.42 High $7.42 The numbers behind this chart are in the table below it.
Where it is a top choice Jobs where deepseek-flash is the best, balanced or cheap choice among the models tested so far.
Math, Easy Best, Balanced, Cheap Thinking low, $0.27 Math, Medium Best, Balanced Thinking low, $0.62 Math, Medium Cheap Thinking off, $0.59 Math, Hard Best, Balanced Thinking low, $2.26 Math, Hard Cheap Thinking off, $1.66 Math, Hardest Best, Balanced Thinking high, $9.18 Math, Hardest Cheap Thinking low, $6.38 SQL and data queries, Easy Best, Balanced, Cheap Thinking off, $0.24 SQL and data queries, Medium Best, Balanced Thinking high, $1.87 SQL and data queries, Medium Cheap Thinking off, $0.45 SQL and data queries, Hard Best, Balanced, Cheap Thinking low, $2.10 SQL and data queries, Hardest Best, Balanced, Cheap Thinking low, $4.29 Stability 7.1%
Different result on a second run. Share of sampled questions that got a different result when asked again.
Compare with Public results for this model One bar per kind of job, from the sources named below. Hover a bar to see the benchmarks behind it, with the score each one reports and whether it was run at settings like ours.
Aggregated from public results Read on 16 November 2026
Zabio has tested this kind of job too. Those results are further up the page and are counted separately.
deepseek-flash Writing, Medium: 76.0 +/- 3.6
Evidence, in standard errors: 1.8
LiveBench Language: 81.2 (Different settings)
LiveBench paraphrase: 70.7 (Different settings)
LMArena text style control, instruction following: 1,475.8 (Different settings) Writing, Medium 76.0 Creative writing, Medium: 72.7 +/- 6.3
Evidence, in standard errors: 1.2
LiveBench story generation: 72.7 (Different settings)
LMArena text style control, creative writing: 1,440.8 (Different settings) Creative writing, Medium 72.7 Summarizing, Medium: 69.7 +/- 6.5
Evidence, in standard errors: -0.0
LiveBench summarize: 69.7 (Different settings) Summarizing, Medium 69.7 Pulling out data, Medium: Rating 1,475.8
Evidence, in standard errors: 1.8
LMArena text style control, instruction following: 1,475.8 (Different settings) Pulling out data, Medium Rating 1,475.8 Sorting into categories, Medium: 70.0 +/- 3.2
Evidence, in standard errors: 0.7
LiveBench IF: 70.0 (Different settings) Sorting into categories, Medium 70.0 Conversation, Medium: 42.9 +/- 1.6
Evidence, in standard errors: 1.0
SimpleQA Verified: 42.9 (Different settings)
LMArena text style control, multi turn: 1,470.6 (Different settings) Conversation, Medium 42.9 Analysis and reasoning, Medium: 66.7 +/- 3.3
Evidence, in standard errors: 0.9
SimpleBench: 66.7 (Same settings as ours) Analysis and reasoning, Medium 66.7 Analysis and reasoning, Hard: 89.8 +/- 1.7
Evidence, in standard errors: 1.9
GPQA diamond: 89.8 (Different settings)
LMArena text style control, hard prompts: 1,498.5 (Different settings) Analysis and reasoning, Hard 89.8 Analysis and reasoning, Hardest: 78.0 +/- 2.3
Evidence, in standard errors: 2.1
ARC-AGI-2: 67.5 (Different settings)
ARC-AGI: 88.5 (Different settings)
LMArena text style control, expert: 1,517.5 (Different settings) Analysis and reasoning, Hardest 78.0 Math, Medium: Rating 1,495.0
Evidence, in standard errors: 1.9
LMArena text style control, math: 1,495.0 (Different settings) Math, Medium Rating 1,495.0 Math, Hard: 93.3 +/- 1.3
Evidence, in standard errors: 2.7
LiveBench Mathematics: 93.3 (Different settings) Math, Hard 93.3 Math, Hardest: 82.9 +/- 1.4
Evidence, in standard errors: 2.4
FrontierMath-Tiers-1-3-v2-Private: 67.4 (Different settings)
OTIS Mock AIME 2024-2025: 98.3 (Different settings) Math, Hardest 82.9 Writing code, Hard: 64.4 +/- 11.4
Evidence, in standard errors: 2.2
MirrorCode: 64.4 (Different settings)
LMArena text style control, coding: 1,525.4 (Different settings)
LMArena webdev, overall: 1,620.8 (Different settings) Writing code, Hard 64.4 Reading images, Medium: Rating 1,279.2
Evidence, in standard errors: 1.0
LMArena vision, overall: 1,279.2 (Different settings) Reading images, Medium Rating 1,279.2 Questions about a document, Medium: Rating 1,480.8
Evidence, in standard errors: 1.3
LMArena text style control, longer query: 1,480.8 (Different settings) Questions about a document, Medium Rating 1,480.8 The benchmarks behind each bar Price against score in each kind of job (15) Price per 1,000 tasks, USD. Score, %. Greyed out: a cheaper model already scores as well. Ringed: the best, balanced and cheap choice.
Writing, Medium 50 60 70 80 90 100 $0.1 $1 $10
deepseek-v4-flash-0731@deepinfra: 70.4 +/- 3.8, $0.12
deepseek-v4-flash@deepinfra: 66.3 +/- 3.9, $0.14
deepseek-v4-flash@parasail: 66.3 +/- 3.9, $0.22
deepseek-v4-flash-0731: 70.4 +/- 3.8, $0.22
deepseek-v4-flash-0731@parasail: 70.4 +/- 3.8, $0.22
deepseek-flash@deepinfra: 76.0 +/- 3.6, $0.28
gpt-6-luna: 61.8 +/- 4.0, $0.28
claude-haiku-5-5: 63.0 +/- 4.0, $0.28
glm-5.3-flash: 62.7 +/- 3.9, $0.32
glm-5.3-flash@deepinfra: 62.7 +/- 3.9, $0.32
deepseek-v4-flash: 66.3 +/- 3.9, $0.32
deepseek-v4-flash-vision-exp: 76.6 +/- 3.5, $0.43
gpt-5.6-luna: 60.2 +/- 4.0, $0.64
gpt-5.4-nano: 64.6 +/- 3.9, $0.66
minimax-m3@deepinfra: 66.4 +/- 3.9, $0.66
deepseek-flash: 76.0 +/- 3.6, $0.72
deepseek-flash@alibaba-intl: 76.0 +/- 3.6, $0.72
deepseek-flash@parasail: 76.0 +/- 3.6, $0.72
deepseek-flash@together: 76.0 +/- 3.6, $0.72
minimax-m3: 66.4 +/- 3.9, $0.72
minimax-m3@parasail: 66.4 +/- 3.9, $0.72
qwen3.8-27b@deepinfra: 74.6 +/- 3.6, $0.87
deepseek-v4-flash-0731@alibaba-intl: 70.4 +/- 3.8, $0.88
qwen3.8-27b@parasail: 74.6 +/- 3.6, $1.07
deepseek-flash-fast@parasail: 76.0 +/- 3.6, $1.08
minimax-m3-fast@parasail: 66.4 +/- 3.9, $1.08
glm-5.2@deepinfra: 69.3 +/- 3.9, $1.17
gemini-3.5-flash-lite: 68.4 +/- 3.8, $1.24
glm-5.3@deepinfra: 72.3 +/- 3.8, $1.45
qwen3.6-27b@deepinfra: 56.4 +/- 4.1, $1.54
qwen3.8-27b: 74.6 +/- 3.6, $1.60
qwen3.6-plus: 67.8 +/- 3.9, $1.60
qwen3.6-27b: 56.4 +/- 4.1, $1.92
kimi-k2.6: 68.4 +/- 3.9, $2.00
deepseek-v4-pro-0813@deepinfra: 75.4 +/- 3.6, $2.08
deepseek-v4-pro@deepinfra: 67.2 +/- 3.9, $2.08
gemini-3.8-flash: 83.9 +/- 3.1, $2.10
gemini-3.7-flash: 81.8 +/- 3.3, $2.10
gemini-3.6-flash: 79.1 +/- 3.4, $2.10
gemini-3.5-flash: 80.1 +/- 3.4, $2.10
kimi-k2.7-code: 63.2 +/- 3.9, $2.36
gpt-5.4-mini: 64.9 +/- 3.9, $2.40
inkling: 72.2 +/- 3.7, $2.42
deepseek-v4-pro-0813: 75.4 +/- 3.6, $2.64
deepseek-v4-pro: 67.2 +/- 3.9, $2.64
deepseek-v4-pro@parasail: 67.2 +/- 3.9, $2.64
deepseek-v4-pro@together: 67.2 +/- 3.9, $2.64
glm-5.3: 72.3 +/- 3.8, $2.88
glm-5.3@alibaba-intl: 72.3 +/- 3.8, $2.88
glm-5.3@parasail: 72.3 +/- 3.8, $2.88
glm-5.2: 69.3 +/- 3.9, $2.88
glm-5.2@alibaba-intl: 69.3 +/- 3.9, $2.88
glm-5.2@alibaba-us: 69.3 +/- 3.9, $2.88
deepseek-v4-pro-fast@parasail: 67.2 +/- 3.9, $3.96
qwen3.8-max-0902: 76.5 +/- 3.5, $4.00
glm-5.3-fast@parasail: 72.3 +/- 3.8, $4.32
grok-4.6: 77.7 +/- 3.5, $4.40
grok-4.7: 79.1 +/- 3.4, $4.40
qwen3.7-max: 75.9 +/- 3.6, $5.00
qwen3.7-max@alibaba-us: 75.9 +/- 3.6, $5.00
gpt-6.1-sol: 81.4 +/- 3.4, $5.60
gpt-6-sol: 77.7 +/- 3.6, $5.60
claude-sonnet-5-5: 63.1 +/- 3.9, $5.60
claude-sonnet-5: 68.1 +/- 3.9, $5.60
gemini-3.1-pro-preview: 82.6 +/- 3.2, $6.40
gpt-5.6-terra: 70.0 +/- 3.8, $6.40
gpt-5.2: 72.0 +/- 3.8, $7.00
kimi-k3@deepinfra: 80.0 +/- 3.4, $7.98
gpt-5.4: 73.0 +/- 3.7, $8.00
kimi-k3: 80.0 +/- 3.4, $8.40
kimi-k3@moonshot: 80.0 +/- 3.4, $8.40
claude-sonnet-4-6: 67.8 +/- 3.9, $8.40
gpt-5.6-sol: 79.6 +/- 3.5, $11.20
claude-opus-5-5: 75.3 +/- 3.7, $11.20
claude-opus-5: 77.2 +/- 3.6, $14.00
claude-opus-4-6: 73.1 +/- 3.7, $14.00
claude-opus-4-7: 71.8 +/- 3.8, $14.00
claude-opus-4-8: 73.8 +/- 3.7, $14.00
claude-opus-4-5: 73.5 +/- 3.7, $14.00
gpt-5.5: 79.6 +/- 3.5, $16.00
claude-fable-5: 84.2 +/- 3.2, $28.00
claude-fable-5-1: 80.5 +/- 3.4, $28.00
gpt-6-astra: 83.8 +/- 3.2, $28.00
deepseek-flash gemini-3.8-flash Cheap claude-fable-5 Best, Balanced Creative writing, Medium 0 25 50 75 100 $0.1 $1 $10
deepseek-v4-flash-0731@deepinfra: 70.5 +/- 6.5, $0.12
deepseek-v4-flash@deepinfra: 67.3 +/- 6.6, $0.14
deepseek-v4-flash@parasail: 67.3 +/- 6.6, $0.22
deepseek-v4-flash-0731: 70.5 +/- 6.5, $0.22
deepseek-v4-flash-0731@parasail: 70.5 +/- 6.5, $0.22
deepseek-flash@deepinfra: 72.7 +/- 6.3, $0.28
gpt-6-luna: 63.3 +/- 6.8, $0.28
claude-haiku-5-5: 73.6 +/- 6.2, $0.28
glm-5.3-flash: 59.9 +/- 6.9, $0.32
glm-5.3-flash@deepinfra: 59.9 +/- 6.9, $0.32
deepseek-v4-flash: 67.3 +/- 6.6, $0.32
deepseek-v4-flash-vision-exp: 73.8 +/- 6.2, $0.43
gpt-5.6-luna: 69.1 +/- 6.5, $0.64
gpt-5.4-nano: 67.7 +/- 6.6, $0.66
minimax-m3@deepinfra: 63.6 +/- 6.8, $0.66
deepseek-flash: 72.7 +/- 6.3, $0.72
deepseek-flash@alibaba-intl: 72.7 +/- 6.3, $0.72
deepseek-flash@parasail: 72.7 +/- 6.3, $0.72
deepseek-flash@together: 72.7 +/- 6.3, $0.72
minimax-m3: 63.6 +/- 6.8, $0.72
minimax-m3@parasail: 63.6 +/- 6.8, $0.72
qwen3.8-27b@deepinfra: 73.1 +/- 6.3, $0.87
deepseek-v4-flash-0731@alibaba-intl: 70.5 +/- 6.5, $0.88
qwen3.8-27b@parasail: 73.1 +/- 6.3, $1.07
deepseek-flash-fast@parasail: 72.7 +/- 6.3, $1.08
minimax-m3-fast@parasail: 63.6 +/- 6.8, $1.08
glm-5.2@deepinfra: 62.9 +/- 6.8, $1.17
gemini-3.5-flash-lite: 67.5 +/- 6.6, $1.24
glm-5.3@deepinfra: 72.1 +/- 6.3, $1.45
qwen3.6-27b@deepinfra: 60.1 +/- 6.9, $1.54
qwen3.6-plus: 56.9 +/- 7.0, $1.60
qwen3.8-27b: 73.1 +/- 6.3, $1.60
qwen3.6-27b: 60.1 +/- 6.9, $1.92
kimi-k2.6: 66.9 +/- 6.7, $2.00
deepseek-v4-pro-0813@deepinfra: 74.9 +/- 6.1, $2.08
deepseek-v4-pro@deepinfra: 67.8 +/- 6.6, $2.08
gemini-3.8-flash: 82.8 +/- 5.3, $2.10
gemini-3.7-flash: 82.3 +/- 5.4, $2.10
gemini-3.6-flash: 75.6 +/- 6.1, $2.10
gemini-3.5-flash: 75.2 +/- 6.1, $2.10
kimi-k2.7-code: 68.6 +/- 6.6, $2.36
gpt-5.4-mini: 66.6 +/- 6.7, $2.40
inkling: 74.2 +/- 6.2, $2.42
deepseek-v4-pro-0813: 74.9 +/- 6.1, $2.64
deepseek-v4-pro: 67.8 +/- 6.6, $2.64
deepseek-v4-pro@parasail: 67.8 +/- 6.6, $2.64
deepseek-v4-pro@together: 67.8 +/- 6.6, $2.64
glm-5.3: 72.1 +/- 6.3, $2.88
glm-5.3@alibaba-intl: 72.1 +/- 6.3, $2.88
glm-5.3@parasail: 72.1 +/- 6.3, $2.88
glm-5.2: 62.9 +/- 6.8, $2.88
glm-5.2@alibaba-intl: 62.9 +/- 6.8, $2.88
glm-5.2@alibaba-us: 62.9 +/- 6.8, $2.88
deepseek-v4-pro-fast@parasail: 67.8 +/- 6.6, $3.96
qwen3.8-max-0902: 76.5 +/- 6.0, $4.00
glm-5.3-fast@parasail: 72.1 +/- 6.3, $4.32
grok-4.6: 78.3 +/- 5.8, $4.40
grok-4.7: 74.9 +/- 6.1, $4.40
qwen3.7-max: 76.3 +/- 6.0, $5.00
qwen3.7-max@alibaba-us: 76.3 +/- 6.0, $5.00
claude-sonnet-5-5: 69.9 +/- 6.5, $5.60
gpt-6.1-sol: 70.8 +/- 6.4, $5.60
gpt-6-sol: 66.9 +/- 6.7, $5.60
claude-sonnet-5: 62.3 +/- 6.9, $5.60
gemini-3.1-pro-preview: 77.3 +/- 5.9, $6.40
gpt-5.6-terra: 66.8 +/- 6.7, $6.40
gpt-5.2: 65.7 +/- 6.7, $7.00
kimi-k3@deepinfra: 75.4 +/- 6.1, $7.98
gpt-5.4: 79.0 +/- 5.8, $8.00
kimi-k3: 75.4 +/- 6.1, $8.40
kimi-k3@moonshot: 75.4 +/- 6.1, $8.40
claude-sonnet-4-6: 68.9 +/- 6.6, $8.40
claude-opus-5-5: 70.9 +/- 6.4, $11.20
gpt-5.6-sol: 75.9 +/- 6.1, $11.20
claude-opus-4-8: 80.6 +/- 5.6, $14.00
claude-opus-4-7: 69.4 +/- 6.5, $14.00
claude-opus-4-6: 66.1 +/- 6.7, $14.00
claude-opus-4-5: 65.9 +/- 6.7, $14.00
claude-opus-5: 61.3 +/- 6.9, $14.00
gpt-5.5: 72.6 +/- 6.3, $16.00
claude-fable-5-1: 78.1 +/- 5.9, $28.00
claude-fable-5: 71.7 +/- 6.4, $28.00
gpt-6-astra: 74.0 +/- 6.2, $28.00
deepseek-v4-flash-0731@deepinfra Cheap deepseek-flash gemini-3.8-flash Best, Balanced Summarizing, Medium 0 25 50 75 100 $0.1 $1 $10
deepseek-v4-flash-0731@deepinfra: 71.5 +/- 6.4, $0.18
deepseek-v4-flash@deepinfra: 68.9 +/- 6.6, $0.25
claude-haiku-5-5: 67.0 +/- 6.7, $0.34
gpt-6-luna: 55.3 +/- 7.0, $0.34
deepseek-v4-flash-0731: 71.5 +/- 6.4, $0.39
deepseek-v4-flash-0731@parasail: 71.5 +/- 6.4, $0.39
deepseek-v4-flash@parasail: 68.9 +/- 6.6, $0.39
deepseek-flash@deepinfra: 69.7 +/- 6.5, $0.42
glm-5.3-flash: 56.5 +/- 7.0, $0.46
glm-5.3-flash@deepinfra: 56.5 +/- 7.0, $0.46
deepseek-v4-flash: 68.9 +/- 6.6, $0.56
deepseek-v4-flash-vision-exp: 72.1 +/- 6.3, $0.65
gpt-5.6-luna: 64.9 +/- 6.8, $0.72
gpt-5.4-nano: 71.8 +/- 6.4, $0.73
qwen3.8-27b@deepinfra: 75.9 +/- 6.1, $0.74
minimax-m3@deepinfra: 53.9 +/- 7.1, $0.89
deepseek-flash: 69.7 +/- 6.5, $0.96
deepseek-flash@alibaba-intl: 69.7 +/- 6.5, $0.96
deepseek-flash@parasail: 69.7 +/- 6.5, $0.96
deepseek-flash@together: 69.7 +/- 6.5, $0.96
minimax-m3: 53.9 +/- 7.1, $0.96
minimax-m3@parasail: 53.9 +/- 7.1, $0.96
qwen3.8-27b@parasail: 75.9 +/- 6.1, $1.02
gemini-3.5-flash-lite: 68.5 +/- 6.6, $1.22
deepseek-v4-flash-0731@alibaba-intl: 71.5 +/- 6.4, $1.32
qwen3.6-27b@deepinfra: 52.6 +/- 7.1, $1.41
deepseek-flash-fast@parasail: 69.7 +/- 6.5, $1.44
minimax-m3-fast@parasail: 53.9 +/- 7.1, $1.44
glm-5.2@deepinfra: 68.4 +/- 6.6, $1.71
qwen3.8-27b: 75.9 +/- 6.1, $1.80
qwen3.6-plus: 64.3 +/- 6.8, $1.80
glm-5.3@deepinfra: 74.6 +/- 6.2, $1.85
qwen3.6-27b: 52.6 +/- 7.1, $2.16
kimi-k2.6: 67.5 +/- 6.6, $2.50
gemini-3.8-flash: 87.5 +/- 4.7, $2.55
gemini-3.7-flash: 84.8 +/- 5.1, $2.55
gemini-3.5-flash: 82.6 +/- 5.4, $2.55
gemini-3.6-flash: 75.9 +/- 6.1, $2.55
gpt-5.4-mini: 59.6 +/- 6.9, $2.70
kimi-k2.7-code: 58.3 +/- 7.0, $3.08
inkling: 74.9 +/- 6.1, $3.21
deepseek-v4-pro@deepinfra: 69.5 +/- 6.5, $3.64
deepseek-v4-pro-0813@deepinfra: 65.7 +/- 6.7, $3.64
deepseek-v4-pro: 69.5 +/- 6.5, $3.96
deepseek-v4-pro@parasail: 69.5 +/- 6.5, $3.96
deepseek-v4-pro@together: 69.5 +/- 6.5, $3.96
deepseek-v4-pro-0813: 65.7 +/- 6.7, $3.96
glm-5.3: 74.6 +/- 6.2, $4.24
glm-5.3@alibaba-intl: 74.6 +/- 6.2, $4.24
glm-5.3@parasail: 74.6 +/- 6.2, $4.24
glm-5.2: 68.4 +/- 6.6, $4.24
glm-5.2@alibaba-intl: 68.4 +/- 6.6, $4.24
glm-5.2@alibaba-us: 68.4 +/- 6.6, $4.24
deepseek-v4-pro-fast@parasail: 69.5 +/- 6.5, $5.94
qwen3.8-max-0902: 79.2 +/- 5.8, $6.00
glm-5.3-fast@parasail: 74.6 +/- 6.2, $6.36
grok-4.7: 78.2 +/- 5.8, $6.60
grok-4.6: 68.1 +/- 6.6, $6.60
gpt-6.1-sol: 81.3 +/- 5.5, $6.80
claude-sonnet-5: 71.2 +/- 6.4, $6.80
gpt-6-sol: 68.0 +/- 6.6, $6.80
claude-sonnet-5-5: 58.2 +/- 7.0, $6.80
gpt-5.2: 63.2 +/- 6.8, $7.00
gemini-3.1-pro-preview: 88.2 +/- 4.6, $7.20
gpt-5.6-terra: 73.2 +/- 6.3, $7.20
qwen3.7-max: 83.8 +/- 5.2, $7.50
qwen3.7-max@alibaba-us: 83.8 +/- 5.2, $7.50
gpt-5.4: 68.4 +/- 6.6, $9.00
kimi-k3@deepinfra: 69.4 +/- 6.5, $9.69
kimi-k3: 69.4 +/- 6.5, $10.20
kimi-k3@moonshot: 69.4 +/- 6.5, $10.20
claude-sonnet-4-6: 61.7 +/- 6.9, $10.20
gpt-5.6-sol: 69.4 +/- 6.5, $13.60
claude-opus-5-5: 69.1 +/- 6.5, $13.60
claude-opus-4-8: 77.7 +/- 5.9, $17.00
claude-opus-5: 66.4 +/- 6.7, $17.00
claude-opus-4-7: 66.3 +/- 6.7, $17.00
claude-opus-4-6: 65.2 +/- 6.7, $17.00
claude-opus-4-5: 63.7 +/- 6.8, $17.00
gpt-5.5: 72.3 +/- 6.3, $18.00
claude-fable-5: 81.7 +/- 5.5, $34.00
gpt-6-astra: 79.6 +/- 5.7, $34.00
claude-fable-5-1: 72.2 +/- 6.3, $34.00
deepseek-v4-flash-vision-exp Cheap qwen3.8-27b@deepinfra Balanced deepseek-flash gemini-3.8-flash Best Pulling out data, Medium 50 60 70 80 90 100 $0.1 $1 $10
deepseek-v4.1-flash-1017@deepinfra: 89.8 +/- 1.7, $0.14
gpt-6-luna: 86.7 +/- 1.9, $0.18
claude-haiku-5-5: 75.6 +/- 2.3, $0.18
glm-5.3-flash: 86.3 +/- 2.2, $0.22
glm-5.3-flash@deepinfra: 86.3 +/- 2.2, $0.22
gemma-4-31b-it: 76.7 +/- 2.1, $0.24
minimax-m3.1-flash: 84.7 +/- 2.1, $0.24
minimax-m3.1-flash@parasail: 84.7 +/- 2.1, $0.24
mimo-v2.6-pro: 85.2 +/- 2.1, $0.47
deepseek-v4.1-flash-1017: 89.8 +/- 1.7, $0.48
deepseek-v4.1-flash-1017@parasail: 89.8 +/- 1.7, $0.48
qwen3.8-27b@deepinfra: 86.4 +/- 2.2, $0.50
qwen3.8-27b@parasail: 86.4 +/- 2.2, $0.63
mistral-large-4-0: 88.3 +/- 1.8, $0.98
qwen3.8-27b: 86.4 +/- 2.2, $1.00
gemini-3.8-flash: 75.1 +/- 2.4, $1.35
inkling: 86.4 +/- 1.9, $1.61
gpt-6.1-terra: 91.2 +/- 1.6, $1.80
gemini-4-hydrogen: 90.6 +/- 1.8, $2.70
qwen3.8-max-0902: 88.2 +/- 1.9, $2.80
grok-4.6: 86.7 +/- 2.0, $3.08
grok-4.7: 88.5 +/- 1.8, $3.08
kimi-k3.1-pro@deepinfra: 88.6 +/- 1.9, $3.42
gemini-4-carbon: 90.3 +/- 2.2, $3.60
gemini-4-argon: 89.7 +/- 2.2, $3.60
gpt-6.1-sol: 91.4 +/- 1.6, $3.60
kimi-k3.1-pro: 88.6 +/- 1.9, $3.60
kimi-k3.1-pro@together: 88.6 +/- 1.9, $3.60
claude-sonnet-5-5: 78.8 +/- 2.2, $3.60
gpt-5.6-terra: 89.7 +/- 1.7, $4.00
kimi-k3@deepinfra: 88.4 +/- 1.9, $5.13
kimi-k3: 88.4 +/- 1.9, $5.40
kimi-k3@moonshot: 88.4 +/- 1.9, $5.40
claude-opus-5-5: 89.2 +/- 1.9, $7.20
claude-fable-5-2: 89.8 +/- 1.9, $14.40
claude-fable-5-1: 87.2 +/- 2.3, $18.00
gpt-6-astra: 91.5 +/- 1.6, $18.00
deepseek-v4.1-flash-1017@deepinfra Cheap glm-5.3-flash Balanced gemini-4-hydrogen Best Sorting into categories, Medium 0 25 50 75 100 $0.1 $1 $10
deepseek-v4-flash-0731@deepinfra: 65.5 +/- 3.4, $0.08
deepseek-v4-flash@deepinfra: 63.1 +/- 3.4, $0.11
deepseek-v4-flash-0731: 65.5 +/- 3.4, $0.17
deepseek-v4-flash-0731@parasail: 65.5 +/- 3.4, $0.17
deepseek-v4-flash@parasail: 63.1 +/- 3.4, $0.17
claude-haiku-5-5: 66.5 +/- 3.3, $0.18
gpt-6-luna: 55.9 +/- 3.5, $0.18
deepseek-flash@deepinfra: 70.0 +/- 3.2, $0.20
glm-5.3-flash: 52.8 +/- 3.5, $0.22
glm-5.3-flash@deepinfra: 52.8 +/- 3.5, $0.22
deepseek-v4-flash: 63.1 +/- 3.4, $0.24
deepseek-v4-flash-vision-exp: 71.0 +/- 3.2, $0.30
gpt-5.6-luna: 60.1 +/- 3.5, $0.40
gpt-5.4-nano: 67.2 +/- 3.3, $0.41
minimax-m3@deepinfra: 57.5 +/- 3.5, $0.44
deepseek-flash: 70.0 +/- 3.2, $0.48
deepseek-flash@alibaba-intl: 70.0 +/- 3.2, $0.48
deepseek-flash@parasail: 70.0 +/- 3.2, $0.48
deepseek-flash@together: 70.0 +/- 3.2, $0.48
minimax-m3: 57.5 +/- 3.5, $0.48
minimax-m3@parasail: 57.5 +/- 3.5, $0.48
qwen3.8-27b@deepinfra: 72.7 +/- 3.2, $0.50
deepseek-v4-flash-0731@alibaba-intl: 65.5 +/- 3.4, $0.62
qwen3.8-27b@parasail: 72.7 +/- 3.2, $0.63
deepseek-flash-fast@parasail: 70.0 +/- 3.2, $0.72
minimax-m3-fast@parasail: 57.5 +/- 3.5, $0.72
gemini-3.5-flash-lite: 67.2 +/- 3.3, $0.74
glm-5.2@deepinfra: 62.3 +/- 3.4, $0.81
qwen3.6-27b@deepinfra: 53.2 +/- 3.5, $0.90
glm-5.3@deepinfra: 69.3 +/- 3.3, $0.95
qwen3.8-27b: 72.7 +/- 3.2, $1.00
qwen3.6-plus: 58.3 +/- 3.5, $1.00
qwen3.6-27b: 53.2 +/- 3.5, $1.20
kimi-k2.6: 64.4 +/- 3.4, $1.30
gemini-3.8-flash: 81.4 +/- 2.8, $1.35
gemini-3.7-flash: 79.9 +/- 2.8, $1.35
gemini-3.5-flash: 75.6 +/- 3.0, $1.35
gemini-3.6-flash: 75.4 +/- 3.1, $1.35
gpt-5.4-mini: 59.8 +/- 3.5, $1.50
deepseek-v4-pro-0813@deepinfra: 67.7 +/- 3.3, $1.56
deepseek-v4-pro@deepinfra: 62.4 +/- 3.4, $1.56
kimi-k2.7-code: 56.3 +/- 3.5, $1.56
inkling: 70.1 +/- 3.2, $1.61
deepseek-v4-pro-0813: 67.7 +/- 3.3, $1.85
deepseek-v4-pro: 62.4 +/- 3.4, $1.85
deepseek-v4-pro@parasail: 62.4 +/- 3.4, $1.85
deepseek-v4-pro@together: 62.4 +/- 3.4, $1.85
glm-5.3: 69.3 +/- 3.3, $2.00
glm-5.3@alibaba-intl: 69.3 +/- 3.3, $2.00
glm-5.3@parasail: 69.3 +/- 3.3, $2.00
glm-5.2: 62.3 +/- 3.4, $2.00
glm-5.2@alibaba-intl: 62.3 +/- 3.4, $2.00
glm-5.2@alibaba-us: 62.3 +/- 3.4, $2.00
deepseek-v4-pro-fast@parasail: 62.4 +/- 3.4, $2.77
qwen3.8-max-0902: 74.1 +/- 3.1, $2.80
glm-5.3-fast@parasail: 69.3 +/- 3.3, $3.00
grok-4.7: 75.3 +/- 3.1, $3.08
grok-4.6: 71.9 +/- 3.2, $3.08
qwen3.7-max: 74.0 +/- 3.1, $3.50
qwen3.7-max@alibaba-us: 74.0 +/- 3.1, $3.50
gpt-6.1-sol: 74.2 +/- 3.1, $3.60
gpt-6-sol: 68.6 +/- 3.3, $3.60
claude-sonnet-5: 63.9 +/- 3.4, $3.60
claude-sonnet-5-5: 56.8 +/- 3.5, $3.60
gemini-3.1-pro-preview: 79.1 +/- 2.9, $4.00
gpt-5.6-terra: 64.6 +/- 3.4, $4.00
gpt-5.2: 61.8 +/- 3.4, $4.20
gpt-5.4: 70.2 +/- 3.2, $5.00
kimi-k3@deepinfra: 71.4 +/- 3.2, $5.13
kimi-k3: 71.4 +/- 3.2, $5.40
kimi-k3@moonshot: 71.4 +/- 3.2, $5.40
claude-sonnet-4-6: 63.2 +/- 3.4, $5.40
gpt-5.6-sol: 71.9 +/- 3.2, $7.20
claude-opus-5-5: 65.7 +/- 3.4, $7.20
claude-opus-4-8: 72.0 +/- 3.2, $9.00
claude-opus-4-7: 66.7 +/- 3.3, $9.00
claude-opus-5: 63.8 +/- 3.4, $9.00
claude-opus-4-6: 63.3 +/- 3.4, $9.00
claude-opus-4-5: 62.6 +/- 3.4, $9.00
gpt-5.5: 70.7 +/- 3.2, $10.00
claude-fable-5: 75.8 +/- 3.0, $18.00
gpt-6-astra: 75.6 +/- 3.0, $18.00
claude-fable-5-1: 73.0 +/- 3.1, $18.00
deepseek-flash gemini-3.8-flash Best, Balanced, Cheap Conversation, Medium 0 25 50 75 100 $0.1 $1 $10
deepseek-v4-flash-0731@deepinfra: 33.6 +/- 1.5, $0.17
gpt-5-nano: 11.7 +/- 1.0, $0.24
deepseek-v4.1-flash-1017@deepinfra: 42.5 +/- 1.6, $0.27
qwen3.5-flash-2026-02-23: 20.3 +/- 1.3, $0.32
gpt-4.1-nano: 6.0 +/- 0.8, $0.32
deepseek-v4-flash-0731: 33.6 +/- 1.5, $0.34
deepseek-v4-flash-0731@parasail: 33.6 +/- 1.5, $0.34
gpt-6-luna: 41.4 +/- 1.6, $0.36
claude-haiku-5-5: 23.8 +/- 1.4, $0.36
deepseek-flash@deepinfra: 42.9 +/- 1.6, $0.39
gemma-4-31b-it: 10.4 +/- 1.0, $0.48
gpt-4o-mini: 8.3 +/- 0.9, $0.48
gpt-5.6-luna: 41.0 +/- 1.6, $0.80
gpt-5.4-nano: 11.7 +/- 1.0, $0.82
deepseek-flash: 42.9 +/- 1.6, $0.96
deepseek-flash@alibaba-intl: 42.9 +/- 1.6, $0.96
deepseek-flash@parasail: 42.9 +/- 1.6, $0.96
deepseek-flash@together: 42.9 +/- 1.6, $0.96
deepseek-v4.1-flash-1017: 42.5 +/- 1.6, $0.96
deepseek-v4.1-flash-1017@parasail: 42.5 +/- 1.6, $0.96
qwen3.6-flash: 15.9 +/- 1.2, $1.00
qwen3.6-flash@alibaba-us: 15.9 +/- 1.2, $1.00
inkling-small: 19.1 +/- 1.2, $1.20
gpt-5-mini: 21.6 +/- 1.3, $1.20
deepseek-v4-flash-0731@alibaba-intl: 33.6 +/- 1.5, $1.23
gpt-4.1-mini: 12.7 +/- 1.1, $1.28
qwen3-235b-a22b-thinking-2507: 40.4 +/- 1.6, $1.29
glm-4.7: 32.2 +/- 1.5, $1.34
deepseek-flash-fast@parasail: 42.9 +/- 1.6, $1.44
glm-5.2@deepinfra: 34.2 +/- 1.5, $1.62
glm-5.3@deepinfra: 41.0 +/- 1.6, $1.90
mistral-large-4-0: 20.0 +/- 1.3, $1.96
gemini-3-flash-preview: 66.8 +/- 1.5, $2.00
qwen3.6-plus: 44.1 +/- 1.6, $2.00
kimi-k2.6: 34.9 +/- 1.5, $2.60
gemini-3.8-flash: 69.7 +/- 1.5, $2.70
gemini-3.7-flash: 69.2 +/- 1.5, $2.70
gemini-3.6-flash: 66.2 +/- 1.5, $2.70
gemini-3.5-flash: 66.2 +/- 1.5, $2.70
gpt-5.4-mini: 29.4 +/- 1.4, $3.00
deepseek-v4-pro-0813@deepinfra: 52.9 +/- 1.6, $3.12
deepseek-v4-pro@deepinfra: 47.0 +/- 1.6, $3.12
kimi-k2.7-code: 36.5 +/- 1.5, $3.12
inkling: 40.3 +/- 1.6, $3.22
o4-mini: 19.6 +/- 1.3, $3.52
o3-mini: 15.3 +/- 1.1, $3.52
gpt-6.1-terra: 57.3 +/- 1.6, $3.60
claude-haiku-4-5: 13.2 +/- 1.1, $3.60
deepseek-v4-pro-0813: 52.9 +/- 1.6, $3.70
deepseek-v4-pro: 47.0 +/- 1.6, $3.70
deepseek-v4-pro@parasail: 47.0 +/- 1.6, $3.70
deepseek-v4-pro@together: 47.0 +/- 1.6, $3.70
glm-5.3: 41.0 +/- 1.6, $4.00
glm-5.3@alibaba-intl: 41.0 +/- 1.6, $4.00
glm-5.3@parasail: 41.0 +/- 1.6, $4.00
glm-5.1: 34.0 +/- 1.5, $4.00
glm-5.2: 34.2 +/- 1.5, $4.00
glm-5.2@alibaba-intl: 34.2 +/- 1.5, $4.00
glm-5.2@alibaba-us: 34.2 +/- 1.5, $4.00
qwen3.6-max-preview: 52.0 +/- 1.6, $5.20
deepseek-v4-pro-fast@parasail: 47.0 +/- 1.6, $5.54
qwen3.8-max-0902: 47.3 +/- 1.6, $5.60
glm-5.3-fast@parasail: 41.0 +/- 1.6, $6.00
gpt-5.1: 48.0 +/- 1.6, $6.00
gpt-5: 50.1 +/- 1.6, $6.00
grok-4.6: 49.3 +/- 1.6, $6.16
grok-4.7: 56.0 +/- 1.6, $6.16
o3: 49.4 +/- 1.6, $6.40
gpt-4.1: 31.1 +/- 1.5, $6.40
qwen3.7-max: 55.8 +/- 1.6, $7.00
qwen3.7-max@alibaba-us: 55.8 +/- 1.6, $7.00
gemini-4-carbon: 77.7 +/- 1.3, $7.20
gemini-4-argon: 76.0 +/- 1.4, $7.20
gpt-6.1-sol: 73.9 +/- 1.4, $7.20
gpt-6-sol: 60.7 +/- 1.6, $7.20
claude-sonnet-5-5: 46.5 +/- 1.6, $7.20
claude-sonnet-5: 33.7 +/- 1.5, $7.20
gemini-3.1-pro-preview: 73.5 +/- 1.4, $8.00
gpt-5.6-terra: 43.2 +/- 1.6, $8.00
gpt-4o: 26.0 +/- 1.4, $8.00
gpt-5.2: 32.8 +/- 1.5, $8.40
gpt-5.4: 45.1 +/- 1.6, $10.00
kimi-k3@deepinfra: 50.6 +/- 1.6, $10.26
kimi-k3: 50.6 +/- 1.6, $10.80
kimi-k3@moonshot: 50.6 +/- 1.6, $10.80
claude-sonnet-4-6: 32.8 +/- 1.5, $10.80
claude-sonnet-4-5: 23.7 +/- 1.4, $10.80
claude-opus-5-5: 72.2 +/- 1.4, $14.40
gpt-5.6-sol: 69.7 +/- 1.5, $14.40
claude-opus-4-7: 51.7 +/- 1.6, $18.00
claude-opus-4-8: 53.0 +/- 1.6, $18.00
claude-opus-5: 59.9 +/- 1.6, $18.00
claude-opus-4-6: 47.0 +/- 1.6, $18.00
claude-opus-4-5: 45.7 +/- 1.6, $18.00
gpt-5.5: 63.0 +/- 1.5, $20.00
claude-fable-5-2: 94.0 +/- 0.8, $28.80
claude-fable-5: 70.7 +/- 1.4, $36.00
gpt-6-astra: 75.6 +/- 1.4, $36.00
claude-fable-5-1: 70.8 +/- 1.4, $36.00
o1: 41.1 +/- 1.6, $48.00
deepseek-flash gemini-4-carbon Best, Balanced, Cheap Analysis and reasoning, Medium 0 25 50 75 100 $0.1 $1 $10
gpt-oss-120b@deepinfra: 22.1 +/- 2.9, $0.10
deepseek-v4-flash-0731@deepinfra: 61.1 +/- 3.5, $0.12
deepseek-v4-flash@deepinfra: 46.3 +/- 3.5, $0.14
llama-3.3-70b-instruct-turbo@deepinfra: 19.9 +/- 2.8, $0.21
deepseek-v4-flash-0731: 61.1 +/- 3.5, $0.22
deepseek-v4-flash-0731@parasail: 61.1 +/- 3.5, $0.22
deepseek-v4-flash@parasail: 46.3 +/- 3.5, $0.22
deepseek-flash@deepinfra: 66.7 +/- 3.3, $0.28
deepseek-v4-flash: 46.3 +/- 3.5, $0.32
gpt-oss-120b: 22.1 +/- 2.9, $0.36
gpt-4o-mini: 10.7 +/- 2.2, $0.36
llama-3.3-70b-instruct-turbo@parasail: 19.9 +/- 2.8, $0.38
gpt-5.6-luna: 46.8 +/- 3.5, $0.64
minimax-m3@deepinfra: 45.8 +/- 3.5, $0.66
deepseek-flash: 66.7 +/- 3.3, $0.72
deepseek-flash@alibaba-intl: 66.7 +/- 3.3, $0.72
deepseek-flash@parasail: 66.7 +/- 3.3, $0.72
deepseek-flash@together: 66.7 +/- 3.3, $0.72
minimax-m3: 45.8 +/- 3.5, $0.72
minimax-m3@parasail: 45.8 +/- 3.5, $0.72
qwen3.6-flash: 35.2 +/- 3.4, $0.80
qwen3.6-flash@alibaba-us: 35.2 +/- 3.4, $0.80
qwen3.8-27b@deepinfra: 60.2 +/- 3.5, $0.87
deepseek-v4-flash-0731@alibaba-intl: 61.1 +/- 3.5, $0.88
mistral-large-2512: 20.4 +/- 2.9, $1.00
glm-4.7: 47.7 +/- 3.5, $1.02
qwen3.8-27b@parasail: 60.2 +/- 3.5, $1.07
deepseek-flash-fast@parasail: 66.7 +/- 3.3, $1.08
minimax-m3-fast@parasail: 45.8 +/- 3.5, $1.08
glm-5.2@deepinfra: 58.8 +/- 3.5, $1.17
llama-3.3-70b-instruct-turbo: 19.9 +/- 2.8, $1.25
deepseek-r1-0528: 40.8 +/- 3.5, $1.26
glm-5.3@deepinfra: 66.2 +/- 3.3, $1.45
gemini-3-flash-preview: 61.1 +/- 3.5, $1.60
qwen3.8-27b: 60.2 +/- 3.5, $1.60
gemini-3.8-flash: 82.4 +/- 2.7, $2.10
gemini-3.5-flash: 76.7 +/- 3.0, $2.10
kimi-k2.7-code: 57.9 +/- 3.5, $2.36
inkling: 50.0 +/- 3.5, $2.42
o4-mini: 38.7 +/- 3.4, $2.64
o3-mini: 22.8 +/- 3.0, $2.64
gpt-6.1-terra: 72.2 +/- 3.2, $2.80
glm-5.3: 66.2 +/- 3.3, $2.88
glm-5.3@alibaba-intl: 66.2 +/- 3.3, $2.88
glm-5.3@parasail: 66.2 +/- 3.3, $2.88
glm-5.2: 58.8 +/- 3.5, $2.88
glm-5.2@alibaba-intl: 58.8 +/- 3.5, $2.88
glm-5.2@alibaba-us: 58.8 +/- 3.5, $2.88
glm-5.1: 55.1 +/- 3.5, $2.88
qwen3.6-max-preview: 63.0 +/- 3.4, $4.16
glm-5.3-fast@parasail: 66.2 +/- 3.3, $4.32
grok-4.6: 75.9 +/- 3.0, $4.40
o3: 53.1 +/- 3.5, $4.80
gpt-4.1: 27.0 +/- 3.1, $4.80
qwen3.7-max: 70.4 +/- 3.2, $5.00
qwen3.7-max@alibaba-us: 70.4 +/- 3.2, $5.00
gpt-5: 56.7 +/- 3.5, $5.00
gpt-5.1: 53.2 +/- 3.5, $5.00
gemini-4-carbon: 92.6 +/- 1.9, $5.60
gemini-4-argon: 91.7 +/- 2.0, $5.60
gpt-6.1-sol: 82.9 +/- 2.7, $5.60
claude-sonnet-5-5: 75.9 +/- 3.0, $5.60
gpt-6-sol: 73.1 +/- 3.1, $5.60
claude-sonnet-5: 60.6 +/- 3.5, $5.60
gpt-4o: 17.8 +/- 2.7, $6.00
gemini-3.1-pro-preview: 79.6 +/- 2.9, $6.40
gpt-5.6-terra: 48.9 +/- 3.5, $6.40
gpt-5.2: 45.8 +/- 3.5, $7.00
kimi-k3@deepinfra: 60.7 +/- 3.5, $7.98
kimi-k3: 60.7 +/- 3.5, $8.40
kimi-k3@moonshot: 60.7 +/- 3.5, $8.40
claude-sonnet-4-5: 54.3 +/- 3.5, $8.40
claude-opus-5-5: 88.4 +/- 2.3, $11.20
gpt-5.6-sol: 71.7 +/- 3.2, $11.20
claude-opus-5: 80.6 +/- 2.8, $14.00
claude-opus-4-6: 67.6 +/- 3.3, $14.00
claude-opus-4-8: 64.8 +/- 3.4, $14.00
claude-opus-4-5: 62.0 +/- 3.4, $14.00
claude-opus-4-7: 61.7 +/- 3.4, $14.00
gpt-5.5: 69.0 +/- 3.3, $16.00
gpt-4-turbo: 25.1 +/- 3.1, $20.00
claude-fable-5-2: 94.9 +/- 1.6, $22.40
claude-fable-5-1: 86.6 +/- 2.4, $28.00
gpt-6-astra: 86.5 +/- 2.4, $28.00
claude-fable-5: 81.9 +/- 2.7, $28.00
o1: 41.7 +/- 3.5, $36.00
deepseek-flash gemini-4-carbon Best, Balanced, Cheap Analysis and reasoning, Hard 0 25 50 75 100 $0.1 $1 $10 $100
gpt-oss-20b: 53.2 +/- 2.5, $0.19
gemma-3-4b-it: 23.2 +/- 1.9, $0.19
gpt-oss-120b@deepinfra: 75.8 +/- 2.7, $0.23
gemma-3-12b-it: 39.5 +/- 2.5, $0.24
phi-4: 56.1 +/- 2.6, $0.27
deepseek-v4-flash-0731@deepinfra: 91.0 +/- 1.8, $0.28
gemma-3-27b-it: 47.7 +/- 2.7, $0.30
qwen3.5-9b@deepinfra: 79.0 +/- 2.4, $0.34
qwen3-32b: 65.7 +/- 2.8, $0.41
qwen3.5-9b@parasail: 79.0 +/- 2.4, $0.43
gemma-4-26b-a4b-it: 73.2 +/- 3.2, $0.45
qwen3-14b: 63.8 +/- 2.9, $0.46
gpt-5-nano: 57.6 +/- 3.5, $0.46
deepseek-v4.1-flash-1017@deepinfra: 87.0 +/- 3.1, $0.46
llama-4-scout-17b-16e-instruct: 51.8 +/- 3.2, $0.47
llama-3.3-70b-instruct-turbo@deepinfra: 47.4 +/- 2.9, $0.49
deepseek-v4-flash-0731: 91.0 +/- 1.8, $0.53
deepseek-v4-flash-0731@parasail: 91.0 +/- 1.8, $0.53
qwen3.5-flash-2026-02-23: 82.3 +/- 2.7, $0.56
gpt-4.1-nano: 48.9 +/- 2.5, $0.56
qwen3.5-9b: 79.0 +/- 2.4, $0.57
gpt-6-luna: 73.6 +/- 3.7, $0.65
claude-haiku-5-5: 85.5 +/- 2.8, $0.65
deepseek-flash@deepinfra: 89.8 +/- 1.7, $0.66
qwen3-30b-a3b: 61.7 +/- 3.0, $0.69
glm-5.3-flash: 80.6 +/- 3.3, $0.75
glm-5.3-flash@deepinfra: 80.6 +/- 3.3, $0.75
gemma-4-31b-it: 61.7 +/- 3.6, $0.76
minimax-m3.1-flash: 79.8 +/- 4.7, $0.84
minimax-m3.1-flash@parasail: 79.8 +/- 4.7, $0.84
gpt-oss-120b: 75.8 +/- 2.7, $0.84
gpt-4o-mini: 37.7 +/- 2.4, $0.84
llama-3.3-70b-instruct-turbo@parasail: 47.4 +/- 2.9, $0.89
qwen3.6-35b-a3b@parasail: 83.8 +/- 2.6, $1.01
qwen3.6-35b-a3b@deepinfra: 83.8 +/- 2.6, $1.06
qwen3-30b-a3b-instruct-2507: 55.6 +/- 2.9, $1.12
qwen3.5-35b-a3b@deepinfra: 83.5 +/- 2.2, $1.18
qwen3.5-35b-a3b@parasail: 83.5 +/- 2.2, $1.20
mimo-v2.6-pro: 82.2 +/- 4.1, $1.48
gpt-5.6-luna: 63.6 +/- 3.4, $1.48
gpt-5.4-nano: 55.6 +/- 3.5, $1.53
minimax-m3@deepinfra: 90.9 +/- 2.1, $1.55
deepseek-v4.1-flash-1017: 87.0 +/- 3.1, $1.68
deepseek-v4.1-flash-1017@parasail: 87.0 +/- 3.1, $1.68
deepseek-flash: 89.8 +/- 1.7, $1.68
deepseek-flash@alibaba-intl: 89.8 +/- 1.7, $1.68
deepseek-flash@parasail: 89.8 +/- 1.7, $1.68
deepseek-flash@together: 89.8 +/- 1.7, $1.68
minimax-m3: 90.9 +/- 2.1, $1.68
minimax-m3@parasail: 90.9 +/- 2.1, $1.68
qwen3.7-plus: 87.9 +/- 2.3, $1.79
qwen3.7-plus@alibaba-us: 87.9 +/- 2.3, $1.79
qwen3.6-flash: 83.3 +/- 2.7, $1.85
qwen3.6-flash@alibaba-us: 83.3 +/- 2.7, $1.85
gemini-3.1-flash-lite: 74.2 +/- 3.1, $1.85
qwen-plus-2025-01-25: 48.1 +/- 2.7, $1.88
inkling-small: 88.5 +/- 1.9, $1.98
qwen3.8-27b@deepinfra: 70.7 +/- 5.4, $1.99
deepseek-v4-flash-0731@alibaba-intl: 91.0 +/- 1.8, $2.07
gpt-4.1-mini: 65.9 +/- 2.7, $2.24
qwen3.5-35b-a3b: 83.5 +/- 2.2, $2.30
gpt-5-mini: 71.7 +/- 3.2, $2.30
gpt-3.5-turbo: 27.2 +/- 2.0, $2.35
glm-4.7: 83.3 +/- 2.4, $2.38
qwen3.8-27b@parasail: 70.7 +/- 5.4, $2.46
deepseek-flash-fast@parasail: 89.8 +/- 1.7, $2.52
minimax-m3-fast@parasail: 90.9 +/- 2.1, $2.52
qwen3-235b-a22b-thinking-2507: 80.1 +/- 2.6, $2.53
qwen3-30b-a3b-thinking-2507: 70.1 +/- 2.8, $2.56
glm-5.2@deepinfra: 71.2 +/- 3.2, $2.75
qwen3.6-35b-a3b: 83.8 +/- 2.6, $2.78
gemini-3.5-flash-lite: 75.8 +/- 3.1, $2.85
deepseek-r1-0528: 76.3 +/- 2.4, $2.94
llama-3.3-70b-instruct-turbo: 47.4 +/- 2.9, $3.02
mistral-large-4-0: 84.5 +/- 3.0, $3.29
glm-5.3@deepinfra: 90.9 +/- 1.6, $3.38
qwen3.6-27b@deepinfra: 85.9 +/- 2.5, $3.52
qwen3.5-397b-a17b@deepinfra: 85.9 +/- 2.5, $3.60
qwen3.6-plus: 88.4 +/- 2.3, $3.70
gemini-3-flash-preview: 83.2 +/- 1.9, $3.70
qwen3.8-27b: 70.7 +/- 5.4, $3.70
qwen3.5-397b-a17b@parasail: 85.9 +/- 2.5, $4.24
qwen3.6-27b: 85.9 +/- 2.5, $4.44
qwen3.5-397b-a17b: 85.9 +/- 2.5, $4.44
kimi-k2.6: 90.8 +/- 1.7, $4.65
gemini-3.8-flash: 88.0 +/- 3.0, $4.88
gemini-3.7-flash: 94.8 +/- 1.4, $4.88
gemini-3.5-flash: 88.9 +/- 2.2, $4.88
gemini-3.6-flash: 85.9 +/- 2.5, $4.88
deepseek-v4-pro-0813@deepinfra: 91.7 +/- 1.5, $4.94
deepseek-v4-pro@deepinfra: 73.2 +/- 3.2, $4.94
kimi-k2.7-code: 87.9 +/- 2.3, $5.50
gpt-5.4-mini: 64.1 +/- 3.4, $5.55
inkling: 75.6 +/- 3.7, $5.65
o4-mini: 75.3 +/- 3.1, $6.16
o3-mini: 68.2 +/- 3.3, $6.16
deepseek-v4-pro-0813: 91.7 +/- 1.5, $6.20
deepseek-v4-pro: 73.2 +/- 3.2, $6.20
deepseek-v4-pro@parasail: 73.2 +/- 3.2, $6.20
deepseek-v4-pro@together: 73.2 +/- 3.2, $6.20
gpt-6.1-terra: 87.4 +/- 3.0, $6.50
claude-haiku-4-5: 60.5 +/- 2.8, $6.50
glm-5.3: 90.9 +/- 1.6, $6.76
glm-5.3@alibaba-intl: 90.9 +/- 1.6, $6.76
glm-5.3@parasail: 90.9 +/- 1.6, $6.76
glm-5.1: 89.9 +/- 2.2, $6.76
glm-5.2: 71.2 +/- 3.2, $6.76
glm-5.2@alibaba-intl: 71.2 +/- 3.2, $6.76
glm-5.2@alibaba-us: 71.2 +/- 3.2, $6.76
deepseek-v4-pro-fast@parasail: 73.2 +/- 3.2, $9.31
qwen3.8-max-0902: 84.4 +/- 3.3, $9.40
qwen3.6-max-preview: 87.4 +/- 2.4, $9.62
gemini-4-hydrogen: 87.7 +/- 3.8, $9.75
glm-5.3-fast@parasail: 90.9 +/- 1.6, $10.14
grok-4.6: 91.1 +/- 2.4, $10.34
grok-4.7: 88.3 +/- 2.4, $10.34
o3: 79.8 +/- 2.9, $11.20
gpt-4.1: 66.9 +/- 2.8, $11.20
gpt-5: 71.7 +/- 3.2, $11.50
gpt-5.1: 66.7 +/- 3.4, $11.50
qwen3.7-max: 90.9 +/- 2.1, $11.75
qwen3.7-max@alibaba-us: 90.9 +/- 2.1, $11.75
kimi-k3.1-pro@deepinfra: 86.6 +/- 4.2, $12.35
gemini-4-argon: 92.8 +/- 2.5, $13.00
gemini-4-carbon: 93.2 +/- 2.0, $13.00
gpt-6.1-sol: 92.7 +/- 2.3, $13.00
claude-sonnet-5-5: 91.7 +/- 2.4, $13.00
gpt-6-sol: 94.3 +/- 1.5, $13.00
kimi-k3.1-pro: 86.6 +/- 4.2, $13.00
kimi-k3.1-pro@together: 86.6 +/- 4.2, $13.00
claude-sonnet-5: 80.3 +/- 2.8, $13.00
gpt-4o: 47.9 +/- 2.6, $14.00
gemini-3.1-pro-preview: 94.1 +/- 1.7, $14.80
gpt-5.6-terra: 81.9 +/- 2.9, $14.80
gpt-5.2: 73.2 +/- 3.2, $16.10
gpt-5.4: 74.8 +/- 3.1, $18.50
kimi-k3@deepinfra: 85.1 +/- 2.9, $18.53
kimi-k3: 85.1 +/- 2.9, $19.50
kimi-k3@moonshot: 85.1 +/- 2.9, $19.50
claude-sonnet-4-6: 78.8 +/- 2.9, $19.50
claude-sonnet-4-5: 73.7 +/- 3.1, $19.50
claude-opus-5-5: 90.6 +/- 2.4, $26.00
gpt-5.6-sol: 82.8 +/- 2.7, $26.00
claude-opus-5: 92.9 +/- 1.8, $32.50
claude-opus-4-6: 88.4 +/- 2.3, $32.50
claude-opus-4-7: 86.4 +/- 2.5, $32.50
claude-opus-4-8: 85.4 +/- 2.5, $32.50
claude-opus-4-5: 80.7 +/- 2.4, $32.50
gpt-5.5: 77.3 +/- 3.0, $37.00
gpt-4-turbo: 46.6 +/- 2.7, $47.00
claude-fable-5-2: 92.4 +/- 1.9, $52.00
gpt-6-astra: 93.0 +/- 2.2, $65.00
claude-fable-5-1: 90.1 +/- 3.5, $65.00
claude-fable-5: 78.8 +/- 2.9, $65.00
o1: 50.3 +/- 2.8, $84.00
gpt-4: 35.7 +/- 2.4, $114
deepseek-flash gemini-3.8-flash Cheap gemini-4-argon Best, Balanced Analysis and reasoning, Hardest 0 25 50 75 100 $1 $10 $100
deepseek-v4-flash-0731@deepinfra: 71.5 +/- 2.4, $0.45
deepseek-v4.1-flash-1017@deepinfra: 81.6 +/- 2.2, $0.74
gpt-5-nano: 9.6 +/- 1.2, $0.75
llama-4-scout-17b-16e-instruct: 0.3 +/- 0.3, $0.75
deepseek-v4-flash-0731: 71.5 +/- 2.4, $0.84
deepseek-v4-flash-0731@parasail: 71.5 +/- 2.4, $0.84
gpt-4.1-nano: 0.0 +/- 0.2, $0.90
deepseek-flash@deepinfra: 78.0 +/- 2.3, $1.05
gpt-6-luna: 50.9 +/- 2.4, $1.05
claude-haiku-5-5: 45.3 +/- 1.6, $1.05
qwen3-235b-a22b-instruct-2507@deepinfra: 6.1 +/- 0.9, $1.10
glm-5.3-flash: 61.0 +/- 2.5, $1.20
glm-5.3-flash@deepinfra: 61.0 +/- 2.5, $1.20
deepseek-v3.2: 30.5 +/- 1.5, $1.35
gpt-4o-mini: 0.0 +/- 0.4, $1.35
qwen3-235b-a22b-instruct-2507@parasail: 6.1 +/- 0.9, $1.62
qwen3-235b-a22b-instruct-2507: 6.1 +/- 0.9, $2.07
inkling-2-small@deepinfra: 89.1 +/- 1.8, $2.28
inkling-2-small: 89.1 +/- 1.8, $2.40
gpt-5.6-luna: 52.9 +/- 2.3, $2.40
gpt-5.4-nano: 20.9 +/- 1.5, $2.48
deepseek-v4.1-flash-1017: 81.6 +/- 2.2, $2.70
deepseek-v4.1-flash-1017@parasail: 81.6 +/- 2.2, $2.70
deepseek-flash: 78.0 +/- 2.3, $2.70
deepseek-flash@alibaba-intl: 78.0 +/- 2.3, $2.70
deepseek-flash@parasail: 78.0 +/- 2.3, $2.70
deepseek-flash@together: 78.0 +/- 2.3, $2.70
gemini-3.1-flash-lite: 8.6 +/- 1.1, $3.00
inkling-small: 55.5 +/- 2.4, $3.15
qwen3.8-27b@deepinfra: 17.8 +/- 1.3, $3.26
deepseek-v4-flash-0731@alibaba-intl: 71.5 +/- 2.4, $3.30
gpt-4.1-mini: 1.8 +/- 0.5, $3.60
gpt-5-mini: 25.2 +/- 1.1, $3.75
qwen3.8-27b@parasail: 17.8 +/- 1.3, $4.02
deepseek-flash-fast@parasail: 78.0 +/- 2.3, $4.05
glm-5.2@deepinfra: 49.9 +/- 2.2, $4.39
gemini-3.5-flash-lite: 31.9 +/- 1.9, $4.65
deepseek-r1-0528: 11.2 +/- 1.1, $4.73
gemini-3-flash-preview: 11.4 +/- 1.1, $6.00
qwen3.8-27b: 17.8 +/- 1.3, $6.00
deepseek-v4-pro-0813@deepinfra: 73.4 +/- 2.4, $7.80
gemini-3.8-flash: 73.3 +/- 1.2, $7.88
gemini-3.7-flash: 90.0 +/- 1.7, $7.88
gemini-3.5-flash: 82.3 +/- 2.2, $7.88
gemini-3.6-flash: 75.8 +/- 2.3, $7.88
gpt-5.4-mini: 35.6 +/- 2.0, $9.00
inkling: 58.0 +/- 2.4, $9.08
deepseek-v4-pro-0813: 73.4 +/- 2.4, $9.90
o4-mini: 27.6 +/- 1.2, $9.90
o3-mini: 18.8 +/- 1.4, $9.90
claude-haiku-4-5: 7.8 +/- 1.0, $10.50
glm-5.2: 49.9 +/- 2.2, $10.80
glm-5.2@alibaba-intl: 49.9 +/- 2.2, $10.80
glm-5.2@alibaba-us: 49.9 +/- 2.2, $10.80
grok-4.6: 76.1 +/- 2.3, $16.50
grok-4.7: 74.3 +/- 2.4, $16.50
o3: 29.2 +/- 1.2, $18.00
gpt-4.1: 3.8 +/- 0.5, $18.00
gpt-5: 32.6 +/- 1.3, $18.75
gpt-5.1: 36.3 +/- 1.4, $18.75
gemini-4-argon: 94.9 +/- 1.3, $21.00
gemini-4-carbon: 94.8 +/- 1.3, $21.00
gpt-6.1-sol: 95.1 +/- 1.3, $21.00
claude-sonnet-5-5: 68.5 +/- 1.6, $21.00
gpt-6-sol: 80.0 +/- 2.2, $21.00
gpt-4o: 2.4 +/- 0.4, $22.50
gemini-3.1-pro-preview: 73.8 +/- 1.5, $24.00
gpt-5.6-terra: 79.5 +/- 2.3, $24.00
gpt-5.2: 47.2 +/- 1.6, $26.25
kimi-k3@deepinfra: 70.8 +/- 2.4, $29.93
gpt-5.4: 65.5 +/- 1.6, $30.00
kimi-k3: 70.8 +/- 2.4, $31.50
kimi-k3@moonshot: 70.8 +/- 2.4, $31.50
claude-sonnet-4-6: 73.5 +/- 2.4, $31.50
claude-sonnet-4-5: 12.3 +/- 1.0, $31.50
claude-opus-5-5: 95.9 +/- 1.2, $42.00
gpt-5.6-sol: 91.2 +/- 1.7, $42.00
claude-opus-5: 92.9 +/- 1.5, $52.50
claude-opus-4-7: 62.3 +/- 1.5, $52.50
claude-opus-4-8: 82.0 +/- 2.2, $52.50
claude-opus-4-6: 51.8 +/- 1.5, $52.50
claude-opus-4-5: 20.7 +/- 1.2, $52.50
gpt-5.5: 88.9 +/- 1.8, $60.00
claude-fable-5-2: 94.0 +/- 1.4, $84.00
claude-fable-5: 91.5 +/- 1.6, $105
claude-fable-5-1: 77.1 +/- 1.2, $105
gpt-6-astra: 78.4 +/- 1.1, $105
o1: 13.0 +/- 1.1, $135
deepseek-flash gemini-4-argon Best, Balanced Math, Medium: Nobody publishes a scored result for this one yet. The models below have a preference rating only.
Math, Hard 0 25 50 75 100 $1 $10 $100
phi-4: 64.9 +/- 1.1, $0.27
deepseek-v4-flash-0731@deepinfra: 86.8 +/- 1.7, $0.28
gemma-3-27b-it: 74.0 +/- 1.1, $0.30
deepseek-v4-flash@deepinfra: 79.7 +/- 2.1, $0.34
gpt-5-nano: 94.9 +/- 0.5, $0.46
llama-4-scout-17b-16e-instruct: 62.3 +/- 1.2, $0.47
llama-3.3-70b-instruct-turbo@deepinfra: 41.6 +/- 1.2, $0.49
deepseek-v4-flash-0731: 86.8 +/- 1.7, $0.53
deepseek-v4-flash-0731@parasail: 86.8 +/- 1.7, $0.53
deepseek-v4-flash@parasail: 79.7 +/- 2.1, $0.53
gpt-4.1-nano: 70.0 +/- 1.0, $0.56
claude-haiku-5-5: 92.8 +/- 1.3, $0.65
gpt-6-luna: 89.1 +/- 1.6, $0.65
deepseek-flash@deepinfra: 93.3 +/- 1.3, $0.66
glm-5.3-flash: 81.2 +/- 2.0, $0.75
glm-5.3-flash@deepinfra: 81.2 +/- 2.0, $0.75
deepseek-v4-flash: 79.7 +/- 2.1, $0.76
gpt-4o-mini: 52.6 +/- 1.1, $0.84
llama-3.3-70b-instruct-turbo@parasail: 41.6 +/- 1.2, $0.89
deepseek-v4-flash-vision-exp: 87.8 +/- 1.7, $1.01
gpt-5.6-luna: 87.2 +/- 1.7, $1.48
gpt-5.4-nano: 91.0 +/- 1.5, $1.53
minimax-m3@deepinfra: 77.0 +/- 2.2, $1.55
deepseek-flash: 93.3 +/- 1.3, $1.68
deepseek-flash@alibaba-intl: 93.3 +/- 1.3, $1.68
deepseek-flash@parasail: 93.3 +/- 1.3, $1.68
deepseek-flash@together: 93.3 +/- 1.3, $1.68
minimax-m3: 77.0 +/- 2.2, $1.68
minimax-m3@parasail: 77.0 +/- 2.2, $1.68
qwen-plus-2025-01-25: 65.3 +/- 1.1, $1.88
qwen3.8-27b@deepinfra: 86.2 +/- 1.8, $1.99
deepseek-v4-flash-0731@alibaba-intl: 86.8 +/- 1.7, $2.07
gpt-4.1-mini: 87.3 +/- 0.7, $2.24
gpt-5-mini: 97.9 +/- 0.3, $2.30
gpt-3.5-turbo: 11.6 +/- 0.6, $2.35
qwen3.8-27b@parasail: 86.2 +/- 1.8, $2.46
deepseek-flash-fast@parasail: 93.3 +/- 1.3, $2.52
minimax-m3-fast@parasail: 77.0 +/- 2.2, $2.52
glm-5.2@deepinfra: 89.8 +/- 1.6, $2.75
gemini-3.5-flash-lite: 73.7 +/- 2.3, $2.85
deepseek-r1-0528: 96.6 +/- 0.4, $2.94
llama-3.3-70b-instruct-turbo: 41.6 +/- 1.2, $3.02
glm-5.3@deepinfra: 87.9 +/- 1.7, $3.38
qwen3.6-27b@deepinfra: 79.9 +/- 2.1, $3.52
qwen3.8-27b: 86.2 +/- 1.8, $3.70
qwen3.6-plus: 83.7 +/- 1.9, $3.70
qwen3.6-27b: 79.9 +/- 2.1, $4.44
kimi-k2.6: 84.3 +/- 1.9, $4.65
gemini-3.7-flash: 93.5 +/- 1.3, $4.88
gemini-3.8-flash: 91.6 +/- 1.4, $4.88
gemini-3.5-flash: 88.2 +/- 1.7, $4.88
gemini-3.6-flash: 86.4 +/- 1.8, $4.88
deepseek-v4-pro-0813@deepinfra: 95.1 +/- 1.1, $4.94
deepseek-v4-pro@deepinfra: 90.7 +/- 1.5, $4.94
kimi-k2.7-code: 79.6 +/- 2.1, $5.50
gpt-5.4-mini: 78.5 +/- 2.1, $5.55
inkling: 88.4 +/- 1.7, $5.65
o4-mini: 97.8 +/- 0.3, $6.16
o3-mini: 96.5 +/- 0.4, $6.16
deepseek-v4-pro-0813: 95.1 +/- 1.1, $6.20
deepseek-v4-pro: 90.7 +/- 1.5, $6.20
deepseek-v4-pro@parasail: 90.7 +/- 1.5, $6.20
deepseek-v4-pro@together: 90.7 +/- 1.5, $6.20
claude-haiku-4-5: 86.9 +/- 0.8, $6.50
glm-5.2: 89.8 +/- 1.6, $6.76
glm-5.2@alibaba-intl: 89.8 +/- 1.6, $6.76
glm-5.2@alibaba-us: 89.8 +/- 1.6, $6.76
glm-5.3: 87.9 +/- 1.7, $6.76
glm-5.3@alibaba-intl: 87.9 +/- 1.7, $6.76
glm-5.3@parasail: 87.9 +/- 1.7, $6.76
deepseek-v4-pro-fast@parasail: 90.7 +/- 1.5, $9.31
qwen3.8-max-0902: 91.3 +/- 1.5, $9.40
glm-5.3-fast@parasail: 87.9 +/- 1.7, $10.14
grok-4.7: 95.7 +/- 1.0, $10.34
grok-4.6: 92.6 +/- 1.4, $10.34
o3: 97.8 +/- 0.3, $11.20
gpt-4.1: 83.0 +/- 0.9, $11.20
gpt-5: 98.1 +/- 0.3, $11.50
qwen3.7-max: 85.3 +/- 1.8, $11.75
qwen3.7-max@alibaba-us: 85.3 +/- 1.8, $11.75
gpt-6.1-sol: 96.8 +/- 0.9, $13.00
gpt-6-sol: 96.4 +/- 1.0, $13.00
claude-sonnet-5-5: 96.1 +/- 1.0, $13.00
claude-sonnet-5: 92.9 +/- 1.3, $13.00
gpt-4o: 49.8 +/- 1.1, $14.00
gpt-5.6-terra: 94.9 +/- 1.1, $14.80
gemini-3.1-pro-preview: 91.0 +/- 1.5, $14.80
gpt-5.2: 93.2 +/- 1.3, $16.10
gpt-5.4: 94.2 +/- 1.2, $18.50
kimi-k3@deepinfra: 84.4 +/- 1.9, $18.53
claude-sonnet-4-5: 97.7 +/- 0.4, $19.50
claude-sonnet-4-6: 87.0 +/- 1.7, $19.50
kimi-k3: 84.4 +/- 1.9, $19.50
kimi-k3@moonshot: 84.4 +/- 1.9, $19.50
claude-opus-5-5: 97.1 +/- 0.9, $26.00
gpt-5.6-sol: 96.2 +/- 1.0, $26.00
claude-opus-5: 95.7 +/- 1.0, $32.50
claude-opus-4-8: 94.3 +/- 1.2, $32.50
claude-opus-4-7: 92.9 +/- 1.3, $32.50
claude-opus-4-5: 90.4 +/- 1.5, $32.50
claude-opus-4-6: 89.3 +/- 1.6, $32.50
gpt-5.5: 95.9 +/- 1.0, $37.00
gpt-4-turbo: 46.7 +/- 1.1, $47.00
claude-fable-5-1: 97.0 +/- 0.9, $65.00
gpt-6-astra: 96.8 +/- 0.9, $65.00
claude-fable-5: 96.0 +/- 1.0, $65.00
o1: 81.7 +/- 1.1, $84.00
gpt-4: 23.0 +/- 0.9, $114
deepseek-flash gpt-5-mini Best, Balanced, Cheap Math, Hardest 0 25 50 75 100 $1 $10 $100
gpt-oss-20b: 40.3 +/- 6.3, $0.30
gemma-3-4b-it: 7.5 +/- 2.8, $0.30
gpt-oss-120b@deepinfra: 88.9 +/- 4.5, $0.37
gemma-3-12b-it: 16.7 +/- 4.2, $0.38
phi-4: 13.8 +/- 3.7, $0.42
deepseek-v4-flash-0731@deepinfra: 76.0 +/- 1.8, $0.45
gemma-3-27b-it: 22.5 +/- 5.3, $0.48
qwen3.5-9b@deepinfra: 42.8 +/- 6.5, $0.53
qwen3-32b: 66.9 +/- 6.0, $0.66
qwen3.5-9b@parasail: 42.8 +/- 6.5, $0.68
gemma-4-26b-a4b-it: 82.2 +/- 5.8, $0.72
qwen3-14b: 66.4 +/- 6.2, $0.72
deepseek-v4.1-flash-1017@deepinfra: 85.3 +/- 1.4, $0.74
gpt-5-nano: 21.1 +/- 2.8, $0.75
llama-4-scout-17b-16e-instruct: 3.9 +/- 2.9, $0.75
llama-3.3-70b-instruct-turbo@deepinfra: 5.1 +/- 2.4, $0.78
deepseek-v4-flash-0731: 76.0 +/- 1.8, $0.84
deepseek-v4-flash-0731@parasail: 76.0 +/- 1.8, $0.84
qwen3.5-9b: 42.8 +/- 6.5, $0.89
qwen3.5-flash-2026-02-23: 33.4 +/- 2.0, $0.90
gpt-4.1-nano: 15.0 +/- 3.0, $0.90
gpt-6-luna: 88.9 +/- 1.3, $1.05
claude-haiku-5-5: 86.2 +/- 1.4, $1.05
deepseek-flash@deepinfra: 82.9 +/- 1.4, $1.05
qwen3-30b-a3b: 62.8 +/- 6.2, $1.11
glm-5.3-flash: 74.8 +/- 1.8, $1.20
glm-5.3-flash@deepinfra: 74.8 +/- 1.8, $1.20
gemma-4-31b-it: 73.3 +/- 6.7, $1.20
gpt-oss-120b: 88.9 +/- 4.5, $1.35
gpt-4o-mini: 3.8 +/- 1.4, $1.35
llama-3.3-70b-instruct-turbo@parasail: 5.1 +/- 2.4, $1.41
qwen3.6-35b-a3b@parasail: 53.5 +/- 2.8, $1.65
qwen3.6-35b-a3b@deepinfra: 53.5 +/- 2.8, $1.73
qwen3-30b-a3b-instruct-2507: 62.2 +/- 6.5, $1.80
qwen3.5-35b-a3b@deepinfra: 54.4 +/- 6.2, $1.92
qwen3.5-35b-a3b@parasail: 54.4 +/- 6.2, $1.95
gpt-5.6-luna: 57.3 +/- 4.4, $2.40
gpt-5.4-nano: 25.7 +/- 2.9, $2.48
minimax-m3@deepinfra: 71.1 +/- 6.8, $2.49
deepseek-v4.1-flash-1017: 85.3 +/- 1.4, $2.70
deepseek-v4.1-flash-1017@parasail: 85.3 +/- 1.4, $2.70
deepseek-flash: 82.9 +/- 1.4, $2.70
deepseek-flash@alibaba-intl: 82.9 +/- 1.4, $2.70
deepseek-flash@parasail: 82.9 +/- 1.4, $2.70
deepseek-flash@together: 82.9 +/- 1.4, $2.70
minimax-m3: 71.1 +/- 6.8, $2.70
minimax-m3@parasail: 71.1 +/- 6.8, $2.70
qwen3.7-plus: 63.9 +/- 2.4, $2.88
qwen3.7-plus@alibaba-us: 63.9 +/- 2.4, $2.88
qwen3.6-flash: 37.3 +/- 2.1, $3.00
qwen3.6-flash@alibaba-us: 37.3 +/- 2.1, $3.00
gemini-3.1-flash-lite: 33.5 +/- 3.9, $3.00
qwen-plus-2025-01-25: 17.8 +/- 4.4, $3.00
qwen-plus: 1.7 +/- 0.8, $3.00
inkling-small: 68.2 +/- 2.5, $3.15
deepseek-v4-flash-0731@alibaba-intl: 76.0 +/- 1.8, $3.30
gpt-4.1-mini: 18.6 +/- 2.2, $3.60
qwen3.5-35b-a3b: 54.4 +/- 6.2, $3.75
gpt-5-mini: 34.1 +/- 2.9, $3.75
gpt-3.5-turbo: 1.1 +/- 2.7, $3.75
glm-4.7: 42.9 +/- 2.4, $3.83
deepseek-flash-fast@parasail: 82.9 +/- 1.4, $4.05
minimax-m3-fast@parasail: 71.1 +/- 6.8, $4.05
qwen3-235b-a22b-thinking-2507: 47.6 +/- 2.7, $4.14
qwen3-30b-a3b-thinking-2507: 70.3 +/- 6.0, $4.20
glm-5.2@deepinfra: 41.4 +/- 4.2, $4.39
qwen3.6-35b-a3b: 53.5 +/- 2.8, $4.50
glm-4.6: 3.8 +/- 1.1, $4.50
gemini-3.5-flash-lite: 44.6 +/- 4.2, $4.65
llama-3.3-70b-instruct-turbo: 5.1 +/- 2.4, $4.68
deepseek-r1-0528: 66.4 +/- 4.9, $4.73
mistral-large-4-0: 78.5 +/- 1.5, $5.25
glm-5.3@deepinfra: 79.9 +/- 2.2, $5.44
qwen3.6-27b@deepinfra: 62.6 +/- 2.6, $5.76
qwen3.5-397b-a17b@deepinfra: 60.1 +/- 2.7, $5.85
gemini-3-flash-preview: 59.9 +/- 1.8, $6.00
qwen3.6-plus: 52.6 +/- 1.8, $6.00
qwen3.5-397b-a17b@parasail: 60.1 +/- 2.7, $6.90
qwen3.6-27b: 62.6 +/- 2.6, $7.20
qwen3.5-397b-a17b: 60.1 +/- 2.7, $7.20
kimi-k2.6: 64.1 +/- 1.6, $7.50
deepseek-v4-pro-0813@deepinfra: 81.6 +/- 1.5, $7.80
deepseek-v4-pro@deepinfra: 46.1 +/- 4.6, $7.80
gemini-3.8-flash: 83.7 +/- 1.5, $7.88
gemini-3.7-flash: 84.4 +/- 1.6, $7.88
gemini-3.5-flash: 65.3 +/- 2.2, $7.88
gemini-3.6-flash: 71.7 +/- 3.4, $7.88
kimi-k2.7-code: 74.8 +/- 2.1, $8.85
gpt-5.4-mini: 23.6 +/- 2.7, $9.00
inkling: 61.1 +/- 2.4, $9.08
deepseek-v4-pro-0813: 81.6 +/- 1.5, $9.90
deepseek-v4-pro: 46.1 +/- 4.6, $9.90
deepseek-v4-pro@parasail: 46.1 +/- 4.6, $9.90
deepseek-v4-pro@together: 46.1 +/- 4.6, $9.90
o4-mini: 28.2 +/- 2.7, $9.90
o3-mini: 20.7 +/- 2.7, $9.90
gpt-6.1-terra: 95.4 +/- 2.6, $10.50
claude-haiku-4-5: 20.0 +/- 3.2, $10.50
glm-5.3: 79.9 +/- 2.2, $10.80
glm-5.3@alibaba-intl: 79.9 +/- 2.2, $10.80
glm-5.3@parasail: 79.9 +/- 2.2, $10.80
glm-5.1: 54.5 +/- 1.8, $10.80
glm-5.2: 41.4 +/- 4.2, $10.80
glm-5.2@alibaba-intl: 41.4 +/- 4.2, $10.80
glm-5.2@alibaba-us: 41.4 +/- 4.2, $10.80
deepseek-v4-pro-fast@parasail: 46.1 +/- 4.6, $14.85
qwen3.8-max-0902: 82.8 +/- 2.9, $15.00
qwen3.6-max-preview: 57.1 +/- 2.5, $15.60
glm-5.3-fast@parasail: 79.9 +/- 2.2, $16.20
grok-4.6: 82.6 +/- 1.4, $16.50
grok-4.7: 75.5 +/- 1.6, $16.50
o3: 29.7 +/- 2.7, $18.00
gpt-4.1: 16.6 +/- 2.2, $18.00
qwen3.7-max: 80.1 +/- 2.1, $18.75
qwen3.7-max@alibaba-us: 80.1 +/- 2.1, $18.75
gpt-5: 39.2 +/- 3.0, $18.75
gpt-5.1: 19.9 +/- 3.7, $18.75
gpt-6.1-sol: 96.8 +/- 2.6, $21.00
gemini-4-carbon: 95.3 +/- 2.7, $21.00
gpt-6-sol: 94.9 +/- 2.7, $21.00
claude-sonnet-5-5: 94.4 +/- 2.7, $21.00
gemini-4-argon: 93.0 +/- 2.7, $21.00
claude-sonnet-5: 72.8 +/- 3.3, $21.00
gpt-4o: 2.3 +/- 0.8, $22.50
gemini-3.1-pro-preview: 64.1 +/- 1.7, $24.00
gpt-5.6-terra: 66.8 +/- 4.5, $24.00
gpt-5.2: 51.8 +/- 3.1, $26.25
kimi-k3@deepinfra: 70.4 +/- 4.0, $29.93
gpt-5.4: 60.9 +/- 3.3, $30.00
kimi-k3: 70.4 +/- 4.0, $31.50
kimi-k3@moonshot: 70.4 +/- 4.0, $31.50
claude-sonnet-4-6: 51.8 +/- 3.7, $31.50
claude-sonnet-4-5: 22.8 +/- 2.8, $31.50
claude-opus-5-5: 95.6 +/- 2.6, $42.00
gpt-5.6-sol: 77.2 +/- 4.2, $42.00
claude-opus-5: 92.8 +/- 1.6, $52.50
claude-opus-4-8: 72.3 +/- 2.5, $52.50
claude-opus-4-7: 66.9 +/- 2.2, $52.50
claude-opus-4-6: 61.8 +/- 1.9, $52.50
claude-opus-4-5: 34.4 +/- 2.6, $52.50
gpt-5.5: 69.1 +/- 4.5, $60.00
gpt-4-turbo: 3.7 +/- 1.2, $75.00
claude-fable-5-2: 97.4 +/- 0.7, $84.00
gpt-6-astra: 96.8 +/- 2.6, $105
claude-fable-5-1: 95.1 +/- 2.7, $105
claude-fable-5: 92.9 +/- 1.5, $105
o1: 17.6 +/- 2.8, $135
gpt-4: 1.1 +/- 0.5, $180
gpt-oss-120b@deepinfra Cheap deepseek-v4.1-flash-1017@deepinfra Balanced deepseek-flash gpt-6.1-sol Best Writing code, Hard 0 25 50 75 100 $1 $10 $100
deepseek-v4.1-flash-1017@deepinfra: 38.8 +/- 11.5, $0.59
deepseek-flash@deepinfra: 64.4 +/- 11.4, $0.85
claude-haiku-5-5: 63.4 +/- 7.1, $0.88
gpt-6-luna: 25.7 +/- 6.8, $0.88
gemma-4-31b-it: 8.5 +/- 6.6, $0.94
glm-5.3-flash: 20.7 +/- 9.6, $0.98
glm-5.3-flash@deepinfra: 20.7 +/- 9.6, $0.98
minimax-m3.1-flash: 25.7 +/- 10.3, $1.11
minimax-m3.1-flash@parasail: 25.7 +/- 10.3, $1.11
mimo-v2.6-pro: 36.4 +/- 11.3, $1.83
gpt-5.6-luna: 21.1 +/- 9.8, $2.02
deepseek-flash: 64.4 +/- 11.4, $2.22
deepseek-flash@alibaba-intl: 64.4 +/- 11.4, $2.22
deepseek-flash@parasail: 64.4 +/- 11.4, $2.22
deepseek-flash@together: 64.4 +/- 11.4, $2.22
deepseek-v4.1-flash-1017: 38.8 +/- 11.5, $2.22
deepseek-v4.1-flash-1017@parasail: 38.8 +/- 11.5, $2.22
qwen3.8-27b@deepinfra: 15.9 +/- 8.6, $2.83
deepseek-flash-fast@parasail: 64.4 +/- 11.4, $3.33
qwen3.8-27b@parasail: 15.9 +/- 8.6, $3.45
glm-5.2@deepinfra: 36.7 +/- 11.5, $3.56
mistral-large-4-0: 32.4 +/- 7.4, $4.24
qwen3.8-27b: 15.9 +/- 8.6, $5.05
deepseek-v4-pro-0813@deepinfra: 33.3 +/- 10.0, $6.11
gemini-3.8-flash: 27.7 +/- 7.4, $6.56
gemini-3.7-flash: 41.1 +/- 12.2, $6.56
gemini-3.6-flash: 23.3 +/- 9.3, $6.56
inkling: 9.8 +/- 7.0, $7.47
deepseek-v4-pro-0813: 33.3 +/- 10.0, $7.99
glm-5.2: 36.7 +/- 11.5, $8.74
glm-5.2@alibaba-intl: 36.7 +/- 11.5, $8.74
glm-5.2@alibaba-us: 36.7 +/- 11.5, $8.74
gpt-6.1-terra: 66.4 +/- 7.5, $8.75
qwen3.8-max-0902: 37.2 +/- 7.3, $12.10
gemini-4-hydrogen: 61.8 +/- 11.5, $13.13
grok-4.7: 49.9 +/- 8.2, $13.31
grok-4.6: 29.9 +/- 7.2, $13.31
gpt-5: 4.4 +/- 3.4, $16.00
kimi-k3.1-pro@deepinfra: 38.7 +/- 11.5, $16.63
gemini-4-carbon: 72.7 +/- 10.5, $17.50
gpt-6.1-sol: 63.4 +/- 8.0, $17.50
gemini-4-argon: 58.7 +/- 8.3, $17.50
claude-sonnet-5-5: 80.6 +/- 5.7, $17.50
kimi-k3.1-pro: 38.7 +/- 11.5, $17.50
kimi-k3.1-pro@together: 38.7 +/- 11.5, $17.50
gpt-6-sol: 48.9 +/- 11.2, $17.50
claude-sonnet-5: 63.3 +/- 10.7, $17.50
gpt-5.6-terra: 33.5 +/- 8.1, $20.20
gemini-3.1-pro-preview: 8.9 +/- 4.6, $20.20
gpt-5.2: 32.2 +/- 10.0, $22.40
kimi-k3@deepinfra: 45.2 +/- 8.0, $24.94
gpt-5.4: 23.3 +/- 8.1, $25.25
kimi-k3: 45.2 +/- 8.0, $26.25
kimi-k3@moonshot: 45.2 +/- 8.0, $26.25
claude-opus-5-5: 84.2 +/- 5.4, $35.00
gpt-5.6-sol: 20.0 +/- 9.2, $35.00
claude-opus-5: 87.8 +/- 7.2, $43.75
claude-opus-4-7: 43.3 +/- 9.6, $43.75
claude-opus-4-8: 65.6 +/- 10.9, $43.75
claude-opus-4-6: 33.3 +/- 10.0, $43.75
gpt-5.5: 23.3 +/- 10.1, $50.50
claude-fable-5-2: 82.9 +/- 8.9, $70.00
claude-fable-5-1: 83.8 +/- 5.5, $87.50
gpt-6-astra: 67.8 +/- 7.8, $87.50
claude-fable-5: 70.0 +/- 10.4, $87.50
deepseek-flash@deepinfra Cheap deepseek-flash gemini-4-carbon Best, Balanced Reading images, Medium: Nobody publishes a scored result for this one yet. The models below have a preference rating only.
Questions about a document, Medium: Nobody publishes a scored result for this one yet. The models below have a preference rating only.
How we tested Called through Zabio's own API at each thinking setting, question set pilot-2026-09-25, methodology 1.0. Without a thinking setting in the request, Zabio uses thinking off.
Model id to use through Zabio: deepseek-flash
Changes 25 September 2026 First results: math and SQL at three thinking settings.