Zabio has not run its own tests on this model. Everything below is aggregated from public benchmarks, with the sources named.
Price against score in each kind of job (5) Price per 1,000 tasks, USD. Score, %. Greyed out: a cheaper model already scores as well. Ringed: the best, balanced and cheap choice.
Writing, Medium 50 60 70 80 90 100 $0.1 $1 $10
deepseek-v4-flash-0731@deepinfra: 70.4 +/- 3.8, $0.12
deepseek-v4-flash@deepinfra: 66.3 +/- 3.9, $0.14
deepseek-v4-flash@parasail: 66.3 +/- 3.9, $0.22
deepseek-v4-flash-0731: 70.4 +/- 3.8, $0.22
deepseek-v4-flash-0731@parasail: 70.4 +/- 3.8, $0.22
deepseek-flash@deepinfra: 76.0 +/- 3.6, $0.28
gpt-6-luna: 61.8 +/- 4.0, $0.28
claude-haiku-5-5: 63.0 +/- 4.0, $0.28
glm-5.3-flash: 62.7 +/- 3.9, $0.32
glm-5.3-flash@deepinfra: 62.7 +/- 3.9, $0.32
deepseek-v4-flash: 66.3 +/- 3.9, $0.32
deepseek-v4-flash-vision-exp: 76.6 +/- 3.5, $0.43
gpt-5.6-luna: 60.2 +/- 4.0, $0.64
gpt-5.4-nano: 64.6 +/- 3.9, $0.66
minimax-m3@deepinfra: 66.4 +/- 3.9, $0.66
deepseek-flash: 76.0 +/- 3.6, $0.72
deepseek-flash@alibaba-intl: 76.0 +/- 3.6, $0.72
deepseek-flash@parasail: 76.0 +/- 3.6, $0.72
deepseek-flash@together: 76.0 +/- 3.6, $0.72
minimax-m3: 66.4 +/- 3.9, $0.72
minimax-m3@parasail: 66.4 +/- 3.9, $0.72
qwen3.8-27b@deepinfra: 74.6 +/- 3.6, $0.87
deepseek-v4-flash-0731@alibaba-intl: 70.4 +/- 3.8, $0.88
qwen3.8-27b@parasail: 74.6 +/- 3.6, $1.07
deepseek-flash-fast@parasail: 76.0 +/- 3.6, $1.08
minimax-m3-fast@parasail: 66.4 +/- 3.9, $1.08
glm-5.2@deepinfra: 69.3 +/- 3.9, $1.17
gemini-3.5-flash-lite: 68.4 +/- 3.8, $1.24
glm-5.3@deepinfra: 72.3 +/- 3.8, $1.45
qwen3.6-27b@deepinfra: 56.4 +/- 4.1, $1.54
qwen3.8-27b: 74.6 +/- 3.6, $1.60
qwen3.6-plus: 67.8 +/- 3.9, $1.60
qwen3.6-27b: 56.4 +/- 4.1, $1.92
kimi-k2.6: 68.4 +/- 3.9, $2.00
deepseek-v4-pro-0813@deepinfra: 75.4 +/- 3.6, $2.08
deepseek-v4-pro@deepinfra: 67.2 +/- 3.9, $2.08
gemini-3.8-flash: 83.9 +/- 3.1, $2.10
gemini-3.7-flash: 81.8 +/- 3.3, $2.10
gemini-3.6-flash: 79.1 +/- 3.4, $2.10
gemini-3.5-flash: 80.1 +/- 3.4, $2.10
kimi-k2.7-code: 63.2 +/- 3.9, $2.36
gpt-5.4-mini: 64.9 +/- 3.9, $2.40
inkling: 72.2 +/- 3.7, $2.42
deepseek-v4-pro-0813: 75.4 +/- 3.6, $2.64
deepseek-v4-pro: 67.2 +/- 3.9, $2.64
deepseek-v4-pro@parasail: 67.2 +/- 3.9, $2.64
deepseek-v4-pro@together: 67.2 +/- 3.9, $2.64
glm-5.3: 72.3 +/- 3.8, $2.88
glm-5.3@alibaba-intl: 72.3 +/- 3.8, $2.88
glm-5.3@parasail: 72.3 +/- 3.8, $2.88
glm-5.2: 69.3 +/- 3.9, $2.88
glm-5.2@alibaba-intl: 69.3 +/- 3.9, $2.88
glm-5.2@alibaba-us: 69.3 +/- 3.9, $2.88
deepseek-v4-pro-fast@parasail: 67.2 +/- 3.9, $3.96
qwen3.8-max-0902: 76.5 +/- 3.5, $4.00
glm-5.3-fast@parasail: 72.3 +/- 3.8, $4.32
grok-4.6: 77.7 +/- 3.5, $4.40
grok-4.7: 79.1 +/- 3.4, $4.40
qwen3.7-max: 75.9 +/- 3.6, $5.00
qwen3.7-max@alibaba-us: 75.9 +/- 3.6, $5.00
gpt-6.1-sol: 81.4 +/- 3.4, $5.60
gpt-6-sol: 77.7 +/- 3.6, $5.60
claude-sonnet-5-5: 63.1 +/- 3.9, $5.60
claude-sonnet-5: 68.1 +/- 3.9, $5.60
gemini-3.1-pro-preview: 82.6 +/- 3.2, $6.40
gpt-5.6-terra: 70.0 +/- 3.8, $6.40
gpt-5.2: 72.0 +/- 3.8, $7.00
kimi-k3@deepinfra: 80.0 +/- 3.4, $7.98
gpt-5.4: 73.0 +/- 3.7, $8.00
kimi-k3: 80.0 +/- 3.4, $8.40
kimi-k3@moonshot: 80.0 +/- 3.4, $8.40
claude-sonnet-4-6: 67.8 +/- 3.9, $8.40
gpt-5.6-sol: 79.6 +/- 3.5, $11.20
claude-opus-5-5: 75.3 +/- 3.7, $11.20
claude-opus-5: 77.2 +/- 3.6, $14.00
claude-opus-4-6: 73.1 +/- 3.7, $14.00
claude-opus-4-7: 71.8 +/- 3.8, $14.00
claude-opus-4-8: 73.8 +/- 3.7, $14.00
claude-opus-4-5: 73.5 +/- 3.7, $14.00
gpt-5.5: 79.6 +/- 3.5, $16.00
claude-fable-5: 84.2 +/- 3.2, $28.00
claude-fable-5-1: 80.5 +/- 3.4, $28.00
gpt-6-astra: 83.8 +/- 3.2, $28.00
deepseek-v4-flash-vision-exp gemini-3.8-flash Cheap claude-fable-5 Best, Balanced Creative writing, Medium 0 25 50 75 100 $0.1 $1 $10
deepseek-v4-flash-0731@deepinfra: 70.5 +/- 6.5, $0.12
deepseek-v4-flash@deepinfra: 67.3 +/- 6.6, $0.14
deepseek-v4-flash@parasail: 67.3 +/- 6.6, $0.22
deepseek-v4-flash-0731: 70.5 +/- 6.5, $0.22
deepseek-v4-flash-0731@parasail: 70.5 +/- 6.5, $0.22
deepseek-flash@deepinfra: 72.7 +/- 6.3, $0.28
gpt-6-luna: 63.3 +/- 6.8, $0.28
claude-haiku-5-5: 73.6 +/- 6.2, $0.28
glm-5.3-flash: 59.9 +/- 6.9, $0.32
glm-5.3-flash@deepinfra: 59.9 +/- 6.9, $0.32
deepseek-v4-flash: 67.3 +/- 6.6, $0.32
deepseek-v4-flash-vision-exp: 73.8 +/- 6.2, $0.43
gpt-5.6-luna: 69.1 +/- 6.5, $0.64
gpt-5.4-nano: 67.7 +/- 6.6, $0.66
minimax-m3@deepinfra: 63.6 +/- 6.8, $0.66
deepseek-flash: 72.7 +/- 6.3, $0.72
deepseek-flash@alibaba-intl: 72.7 +/- 6.3, $0.72
deepseek-flash@parasail: 72.7 +/- 6.3, $0.72
deepseek-flash@together: 72.7 +/- 6.3, $0.72
minimax-m3: 63.6 +/- 6.8, $0.72
minimax-m3@parasail: 63.6 +/- 6.8, $0.72
qwen3.8-27b@deepinfra: 73.1 +/- 6.3, $0.87
deepseek-v4-flash-0731@alibaba-intl: 70.5 +/- 6.5, $0.88
qwen3.8-27b@parasail: 73.1 +/- 6.3, $1.07
deepseek-flash-fast@parasail: 72.7 +/- 6.3, $1.08
minimax-m3-fast@parasail: 63.6 +/- 6.8, $1.08
glm-5.2@deepinfra: 62.9 +/- 6.8, $1.17
gemini-3.5-flash-lite: 67.5 +/- 6.6, $1.24
glm-5.3@deepinfra: 72.1 +/- 6.3, $1.45
qwen3.6-27b@deepinfra: 60.1 +/- 6.9, $1.54
qwen3.6-plus: 56.9 +/- 7.0, $1.60
qwen3.8-27b: 73.1 +/- 6.3, $1.60
qwen3.6-27b: 60.1 +/- 6.9, $1.92
kimi-k2.6: 66.9 +/- 6.7, $2.00
deepseek-v4-pro-0813@deepinfra: 74.9 +/- 6.1, $2.08
deepseek-v4-pro@deepinfra: 67.8 +/- 6.6, $2.08
gemini-3.8-flash: 82.8 +/- 5.3, $2.10
gemini-3.7-flash: 82.3 +/- 5.4, $2.10
gemini-3.6-flash: 75.6 +/- 6.1, $2.10
gemini-3.5-flash: 75.2 +/- 6.1, $2.10
kimi-k2.7-code: 68.6 +/- 6.6, $2.36
gpt-5.4-mini: 66.6 +/- 6.7, $2.40
inkling: 74.2 +/- 6.2, $2.42
deepseek-v4-pro-0813: 74.9 +/- 6.1, $2.64
deepseek-v4-pro: 67.8 +/- 6.6, $2.64
deepseek-v4-pro@parasail: 67.8 +/- 6.6, $2.64
deepseek-v4-pro@together: 67.8 +/- 6.6, $2.64
glm-5.3: 72.1 +/- 6.3, $2.88
glm-5.3@alibaba-intl: 72.1 +/- 6.3, $2.88
glm-5.3@parasail: 72.1 +/- 6.3, $2.88
glm-5.2: 62.9 +/- 6.8, $2.88
glm-5.2@alibaba-intl: 62.9 +/- 6.8, $2.88
glm-5.2@alibaba-us: 62.9 +/- 6.8, $2.88
deepseek-v4-pro-fast@parasail: 67.8 +/- 6.6, $3.96
qwen3.8-max-0902: 76.5 +/- 6.0, $4.00
glm-5.3-fast@parasail: 72.1 +/- 6.3, $4.32
grok-4.6: 78.3 +/- 5.8, $4.40
grok-4.7: 74.9 +/- 6.1, $4.40
qwen3.7-max: 76.3 +/- 6.0, $5.00
qwen3.7-max@alibaba-us: 76.3 +/- 6.0, $5.00
claude-sonnet-5-5: 69.9 +/- 6.5, $5.60
gpt-6.1-sol: 70.8 +/- 6.4, $5.60
gpt-6-sol: 66.9 +/- 6.7, $5.60
claude-sonnet-5: 62.3 +/- 6.9, $5.60
gemini-3.1-pro-preview: 77.3 +/- 5.9, $6.40
gpt-5.6-terra: 66.8 +/- 6.7, $6.40
gpt-5.2: 65.7 +/- 6.7, $7.00
kimi-k3@deepinfra: 75.4 +/- 6.1, $7.98
gpt-5.4: 79.0 +/- 5.8, $8.00
kimi-k3: 75.4 +/- 6.1, $8.40
kimi-k3@moonshot: 75.4 +/- 6.1, $8.40
claude-sonnet-4-6: 68.9 +/- 6.6, $8.40
claude-opus-5-5: 70.9 +/- 6.4, $11.20
gpt-5.6-sol: 75.9 +/- 6.1, $11.20
claude-opus-4-8: 80.6 +/- 5.6, $14.00
claude-opus-4-7: 69.4 +/- 6.5, $14.00
claude-opus-4-6: 66.1 +/- 6.7, $14.00
claude-opus-4-5: 65.9 +/- 6.7, $14.00
claude-opus-5: 61.3 +/- 6.9, $14.00
gpt-5.5: 72.6 +/- 6.3, $16.00
claude-fable-5-1: 78.1 +/- 5.9, $28.00
claude-fable-5: 71.7 +/- 6.4, $28.00
gpt-6-astra: 74.0 +/- 6.2, $28.00
deepseek-v4-flash-0731@deepinfra Cheap deepseek-v4-flash-vision-exp gemini-3.8-flash Best, Balanced Summarizing, Medium 0 25 50 75 100 $0.1 $1 $10
deepseek-v4-flash-0731@deepinfra: 71.5 +/- 6.4, $0.18
deepseek-v4-flash@deepinfra: 68.9 +/- 6.6, $0.25
claude-haiku-5-5: 67.0 +/- 6.7, $0.34
gpt-6-luna: 55.3 +/- 7.0, $0.34
deepseek-v4-flash-0731: 71.5 +/- 6.4, $0.39
deepseek-v4-flash-0731@parasail: 71.5 +/- 6.4, $0.39
deepseek-v4-flash@parasail: 68.9 +/- 6.6, $0.39
deepseek-flash@deepinfra: 69.7 +/- 6.5, $0.42
glm-5.3-flash: 56.5 +/- 7.0, $0.46
glm-5.3-flash@deepinfra: 56.5 +/- 7.0, $0.46
deepseek-v4-flash: 68.9 +/- 6.6, $0.56
deepseek-v4-flash-vision-exp: 72.1 +/- 6.3, $0.65
gpt-5.6-luna: 64.9 +/- 6.8, $0.72
gpt-5.4-nano: 71.8 +/- 6.4, $0.73
qwen3.8-27b@deepinfra: 75.9 +/- 6.1, $0.74
minimax-m3@deepinfra: 53.9 +/- 7.1, $0.89
deepseek-flash: 69.7 +/- 6.5, $0.96
deepseek-flash@alibaba-intl: 69.7 +/- 6.5, $0.96
deepseek-flash@parasail: 69.7 +/- 6.5, $0.96
deepseek-flash@together: 69.7 +/- 6.5, $0.96
minimax-m3: 53.9 +/- 7.1, $0.96
minimax-m3@parasail: 53.9 +/- 7.1, $0.96
qwen3.8-27b@parasail: 75.9 +/- 6.1, $1.02
gemini-3.5-flash-lite: 68.5 +/- 6.6, $1.22
deepseek-v4-flash-0731@alibaba-intl: 71.5 +/- 6.4, $1.32
qwen3.6-27b@deepinfra: 52.6 +/- 7.1, $1.41
deepseek-flash-fast@parasail: 69.7 +/- 6.5, $1.44
minimax-m3-fast@parasail: 53.9 +/- 7.1, $1.44
glm-5.2@deepinfra: 68.4 +/- 6.6, $1.71
qwen3.8-27b: 75.9 +/- 6.1, $1.80
qwen3.6-plus: 64.3 +/- 6.8, $1.80
glm-5.3@deepinfra: 74.6 +/- 6.2, $1.85
qwen3.6-27b: 52.6 +/- 7.1, $2.16
kimi-k2.6: 67.5 +/- 6.6, $2.50
gemini-3.8-flash: 87.5 +/- 4.7, $2.55
gemini-3.7-flash: 84.8 +/- 5.1, $2.55
gemini-3.5-flash: 82.6 +/- 5.4, $2.55
gemini-3.6-flash: 75.9 +/- 6.1, $2.55
gpt-5.4-mini: 59.6 +/- 6.9, $2.70
kimi-k2.7-code: 58.3 +/- 7.0, $3.08
inkling: 74.9 +/- 6.1, $3.21
deepseek-v4-pro@deepinfra: 69.5 +/- 6.5, $3.64
deepseek-v4-pro-0813@deepinfra: 65.7 +/- 6.7, $3.64
deepseek-v4-pro: 69.5 +/- 6.5, $3.96
deepseek-v4-pro@parasail: 69.5 +/- 6.5, $3.96
deepseek-v4-pro@together: 69.5 +/- 6.5, $3.96
deepseek-v4-pro-0813: 65.7 +/- 6.7, $3.96
glm-5.3: 74.6 +/- 6.2, $4.24
glm-5.3@alibaba-intl: 74.6 +/- 6.2, $4.24
glm-5.3@parasail: 74.6 +/- 6.2, $4.24
glm-5.2: 68.4 +/- 6.6, $4.24
glm-5.2@alibaba-intl: 68.4 +/- 6.6, $4.24
glm-5.2@alibaba-us: 68.4 +/- 6.6, $4.24
deepseek-v4-pro-fast@parasail: 69.5 +/- 6.5, $5.94
qwen3.8-max-0902: 79.2 +/- 5.8, $6.00
glm-5.3-fast@parasail: 74.6 +/- 6.2, $6.36
grok-4.7: 78.2 +/- 5.8, $6.60
grok-4.6: 68.1 +/- 6.6, $6.60
gpt-6.1-sol: 81.3 +/- 5.5, $6.80
claude-sonnet-5: 71.2 +/- 6.4, $6.80
gpt-6-sol: 68.0 +/- 6.6, $6.80
claude-sonnet-5-5: 58.2 +/- 7.0, $6.80
gpt-5.2: 63.2 +/- 6.8, $7.00
gemini-3.1-pro-preview: 88.2 +/- 4.6, $7.20
gpt-5.6-terra: 73.2 +/- 6.3, $7.20
qwen3.7-max: 83.8 +/- 5.2, $7.50
qwen3.7-max@alibaba-us: 83.8 +/- 5.2, $7.50
gpt-5.4: 68.4 +/- 6.6, $9.00
kimi-k3@deepinfra: 69.4 +/- 6.5, $9.69
kimi-k3: 69.4 +/- 6.5, $10.20
kimi-k3@moonshot: 69.4 +/- 6.5, $10.20
claude-sonnet-4-6: 61.7 +/- 6.9, $10.20
gpt-5.6-sol: 69.4 +/- 6.5, $13.60
claude-opus-5-5: 69.1 +/- 6.5, $13.60
claude-opus-4-8: 77.7 +/- 5.9, $17.00
claude-opus-5: 66.4 +/- 6.7, $17.00
claude-opus-4-7: 66.3 +/- 6.7, $17.00
claude-opus-4-6: 65.2 +/- 6.7, $17.00
claude-opus-4-5: 63.7 +/- 6.8, $17.00
gpt-5.5: 72.3 +/- 6.3, $18.00
claude-fable-5: 81.7 +/- 5.5, $34.00
gpt-6-astra: 79.6 +/- 5.7, $34.00
claude-fable-5-1: 72.2 +/- 6.3, $34.00
deepseek-v4-flash-vision-exp Cheap qwen3.8-27b@deepinfra Balanced gemini-3.8-flash Best Sorting into categories, Medium 0 25 50 75 100 $0.1 $1 $10
deepseek-v4-flash-0731@deepinfra: 65.5 +/- 3.4, $0.08
deepseek-v4-flash@deepinfra: 63.1 +/- 3.4, $0.11
deepseek-v4-flash-0731: 65.5 +/- 3.4, $0.17
deepseek-v4-flash-0731@parasail: 65.5 +/- 3.4, $0.17
deepseek-v4-flash@parasail: 63.1 +/- 3.4, $0.17
claude-haiku-5-5: 66.5 +/- 3.3, $0.18
gpt-6-luna: 55.9 +/- 3.5, $0.18
deepseek-flash@deepinfra: 70.0 +/- 3.2, $0.20
glm-5.3-flash: 52.8 +/- 3.5, $0.22
glm-5.3-flash@deepinfra: 52.8 +/- 3.5, $0.22
deepseek-v4-flash: 63.1 +/- 3.4, $0.24
deepseek-v4-flash-vision-exp: 71.0 +/- 3.2, $0.30
gpt-5.6-luna: 60.1 +/- 3.5, $0.40
gpt-5.4-nano: 67.2 +/- 3.3, $0.41
minimax-m3@deepinfra: 57.5 +/- 3.5, $0.44
deepseek-flash: 70.0 +/- 3.2, $0.48
deepseek-flash@alibaba-intl: 70.0 +/- 3.2, $0.48
deepseek-flash@parasail: 70.0 +/- 3.2, $0.48
deepseek-flash@together: 70.0 +/- 3.2, $0.48
minimax-m3: 57.5 +/- 3.5, $0.48
minimax-m3@parasail: 57.5 +/- 3.5, $0.48
qwen3.8-27b@deepinfra: 72.7 +/- 3.2, $0.50
deepseek-v4-flash-0731@alibaba-intl: 65.5 +/- 3.4, $0.62
qwen3.8-27b@parasail: 72.7 +/- 3.2, $0.63
deepseek-flash-fast@parasail: 70.0 +/- 3.2, $0.72
minimax-m3-fast@parasail: 57.5 +/- 3.5, $0.72
gemini-3.5-flash-lite: 67.2 +/- 3.3, $0.74
glm-5.2@deepinfra: 62.3 +/- 3.4, $0.81
qwen3.6-27b@deepinfra: 53.2 +/- 3.5, $0.90
glm-5.3@deepinfra: 69.3 +/- 3.3, $0.95
qwen3.8-27b: 72.7 +/- 3.2, $1.00
qwen3.6-plus: 58.3 +/- 3.5, $1.00
qwen3.6-27b: 53.2 +/- 3.5, $1.20
kimi-k2.6: 64.4 +/- 3.4, $1.30
gemini-3.8-flash: 81.4 +/- 2.8, $1.35
gemini-3.7-flash: 79.9 +/- 2.8, $1.35
gemini-3.5-flash: 75.6 +/- 3.0, $1.35
gemini-3.6-flash: 75.4 +/- 3.1, $1.35
gpt-5.4-mini: 59.8 +/- 3.5, $1.50
deepseek-v4-pro-0813@deepinfra: 67.7 +/- 3.3, $1.56
deepseek-v4-pro@deepinfra: 62.4 +/- 3.4, $1.56
kimi-k2.7-code: 56.3 +/- 3.5, $1.56
inkling: 70.1 +/- 3.2, $1.61
deepseek-v4-pro-0813: 67.7 +/- 3.3, $1.85
deepseek-v4-pro: 62.4 +/- 3.4, $1.85
deepseek-v4-pro@parasail: 62.4 +/- 3.4, $1.85
deepseek-v4-pro@together: 62.4 +/- 3.4, $1.85
glm-5.3: 69.3 +/- 3.3, $2.00
glm-5.3@alibaba-intl: 69.3 +/- 3.3, $2.00
glm-5.3@parasail: 69.3 +/- 3.3, $2.00
glm-5.2: 62.3 +/- 3.4, $2.00
glm-5.2@alibaba-intl: 62.3 +/- 3.4, $2.00
glm-5.2@alibaba-us: 62.3 +/- 3.4, $2.00
deepseek-v4-pro-fast@parasail: 62.4 +/- 3.4, $2.77
qwen3.8-max-0902: 74.1 +/- 3.1, $2.80
glm-5.3-fast@parasail: 69.3 +/- 3.3, $3.00
grok-4.7: 75.3 +/- 3.1, $3.08
grok-4.6: 71.9 +/- 3.2, $3.08
qwen3.7-max: 74.0 +/- 3.1, $3.50
qwen3.7-max@alibaba-us: 74.0 +/- 3.1, $3.50
gpt-6.1-sol: 74.2 +/- 3.1, $3.60
gpt-6-sol: 68.6 +/- 3.3, $3.60
claude-sonnet-5: 63.9 +/- 3.4, $3.60
claude-sonnet-5-5: 56.8 +/- 3.5, $3.60
gemini-3.1-pro-preview: 79.1 +/- 2.9, $4.00
gpt-5.6-terra: 64.6 +/- 3.4, $4.00
gpt-5.2: 61.8 +/- 3.4, $4.20
gpt-5.4: 70.2 +/- 3.2, $5.00
kimi-k3@deepinfra: 71.4 +/- 3.2, $5.13
kimi-k3: 71.4 +/- 3.2, $5.40
kimi-k3@moonshot: 71.4 +/- 3.2, $5.40
claude-sonnet-4-6: 63.2 +/- 3.4, $5.40
gpt-5.6-sol: 71.9 +/- 3.2, $7.20
claude-opus-5-5: 65.7 +/- 3.4, $7.20
claude-opus-4-8: 72.0 +/- 3.2, $9.00
claude-opus-4-7: 66.7 +/- 3.3, $9.00
claude-opus-5: 63.8 +/- 3.4, $9.00
claude-opus-4-6: 63.3 +/- 3.4, $9.00
claude-opus-4-5: 62.6 +/- 3.4, $9.00
gpt-5.5: 70.7 +/- 3.2, $10.00
claude-fable-5: 75.8 +/- 3.0, $18.00
gpt-6-astra: 75.6 +/- 3.0, $18.00
claude-fable-5-1: 73.0 +/- 3.1, $18.00
deepseek-v4-flash-vision-exp gemini-3.8-flash Best, Balanced, Cheap Math, Hard 0 25 50 75 100 $1 $10 $100
phi-4: 64.9 +/- 1.1, $0.27
deepseek-v4-flash-0731@deepinfra: 86.8 +/- 1.7, $0.28
gemma-3-27b-it: 74.0 +/- 1.1, $0.30
deepseek-v4-flash@deepinfra: 79.7 +/- 2.1, $0.34
gpt-5-nano: 94.9 +/- 0.5, $0.46
llama-4-scout-17b-16e-instruct: 62.3 +/- 1.2, $0.47
llama-3.3-70b-instruct-turbo@deepinfra: 41.6 +/- 1.2, $0.49
deepseek-v4-flash-0731: 86.8 +/- 1.7, $0.53
deepseek-v4-flash-0731@parasail: 86.8 +/- 1.7, $0.53
deepseek-v4-flash@parasail: 79.7 +/- 2.1, $0.53
gpt-4.1-nano: 70.0 +/- 1.0, $0.56
claude-haiku-5-5: 92.8 +/- 1.3, $0.65
gpt-6-luna: 89.1 +/- 1.6, $0.65
deepseek-flash@deepinfra: 93.3 +/- 1.3, $0.66
glm-5.3-flash: 81.2 +/- 2.0, $0.75
glm-5.3-flash@deepinfra: 81.2 +/- 2.0, $0.75
deepseek-v4-flash: 79.7 +/- 2.1, $0.76
gpt-4o-mini: 52.6 +/- 1.1, $0.84
llama-3.3-70b-instruct-turbo@parasail: 41.6 +/- 1.2, $0.89
deepseek-v4-flash-vision-exp: 87.8 +/- 1.7, $1.01
gpt-5.6-luna: 87.2 +/- 1.7, $1.48
gpt-5.4-nano: 91.0 +/- 1.5, $1.53
minimax-m3@deepinfra: 77.0 +/- 2.2, $1.55
deepseek-flash: 93.3 +/- 1.3, $1.68
deepseek-flash@alibaba-intl: 93.3 +/- 1.3, $1.68
deepseek-flash@parasail: 93.3 +/- 1.3, $1.68
deepseek-flash@together: 93.3 +/- 1.3, $1.68
minimax-m3: 77.0 +/- 2.2, $1.68
minimax-m3@parasail: 77.0 +/- 2.2, $1.68
qwen-plus-2025-01-25: 65.3 +/- 1.1, $1.88
qwen3.8-27b@deepinfra: 86.2 +/- 1.8, $1.99
deepseek-v4-flash-0731@alibaba-intl: 86.8 +/- 1.7, $2.07
gpt-4.1-mini: 87.3 +/- 0.7, $2.24
gpt-5-mini: 97.9 +/- 0.3, $2.30
gpt-3.5-turbo: 11.6 +/- 0.6, $2.35
qwen3.8-27b@parasail: 86.2 +/- 1.8, $2.46
deepseek-flash-fast@parasail: 93.3 +/- 1.3, $2.52
minimax-m3-fast@parasail: 77.0 +/- 2.2, $2.52
glm-5.2@deepinfra: 89.8 +/- 1.6, $2.75
gemini-3.5-flash-lite: 73.7 +/- 2.3, $2.85
deepseek-r1-0528: 96.6 +/- 0.4, $2.94
llama-3.3-70b-instruct-turbo: 41.6 +/- 1.2, $3.02
glm-5.3@deepinfra: 87.9 +/- 1.7, $3.38
qwen3.6-27b@deepinfra: 79.9 +/- 2.1, $3.52
qwen3.8-27b: 86.2 +/- 1.8, $3.70
qwen3.6-plus: 83.7 +/- 1.9, $3.70
qwen3.6-27b: 79.9 +/- 2.1, $4.44
kimi-k2.6: 84.3 +/- 1.9, $4.65
gemini-3.7-flash: 93.5 +/- 1.3, $4.88
gemini-3.8-flash: 91.6 +/- 1.4, $4.88
gemini-3.5-flash: 88.2 +/- 1.7, $4.88
gemini-3.6-flash: 86.4 +/- 1.8, $4.88
deepseek-v4-pro-0813@deepinfra: 95.1 +/- 1.1, $4.94
deepseek-v4-pro@deepinfra: 90.7 +/- 1.5, $4.94
kimi-k2.7-code: 79.6 +/- 2.1, $5.50
gpt-5.4-mini: 78.5 +/- 2.1, $5.55
inkling: 88.4 +/- 1.7, $5.65
o4-mini: 97.8 +/- 0.3, $6.16
o3-mini: 96.5 +/- 0.4, $6.16
deepseek-v4-pro-0813: 95.1 +/- 1.1, $6.20
deepseek-v4-pro: 90.7 +/- 1.5, $6.20
deepseek-v4-pro@parasail: 90.7 +/- 1.5, $6.20
deepseek-v4-pro@together: 90.7 +/- 1.5, $6.20
claude-haiku-4-5: 86.9 +/- 0.8, $6.50
glm-5.2: 89.8 +/- 1.6, $6.76
glm-5.2@alibaba-intl: 89.8 +/- 1.6, $6.76
glm-5.2@alibaba-us: 89.8 +/- 1.6, $6.76
glm-5.3: 87.9 +/- 1.7, $6.76
glm-5.3@alibaba-intl: 87.9 +/- 1.7, $6.76
glm-5.3@parasail: 87.9 +/- 1.7, $6.76
deepseek-v4-pro-fast@parasail: 90.7 +/- 1.5, $9.31
qwen3.8-max-0902: 91.3 +/- 1.5, $9.40
glm-5.3-fast@parasail: 87.9 +/- 1.7, $10.14
grok-4.7: 95.7 +/- 1.0, $10.34
grok-4.6: 92.6 +/- 1.4, $10.34
o3: 97.8 +/- 0.3, $11.20
gpt-4.1: 83.0 +/- 0.9, $11.20
gpt-5: 98.1 +/- 0.3, $11.50
qwen3.7-max: 85.3 +/- 1.8, $11.75
qwen3.7-max@alibaba-us: 85.3 +/- 1.8, $11.75
gpt-6.1-sol: 96.8 +/- 0.9, $13.00
gpt-6-sol: 96.4 +/- 1.0, $13.00
claude-sonnet-5-5: 96.1 +/- 1.0, $13.00
claude-sonnet-5: 92.9 +/- 1.3, $13.00
gpt-4o: 49.8 +/- 1.1, $14.00
gpt-5.6-terra: 94.9 +/- 1.1, $14.80
gemini-3.1-pro-preview: 91.0 +/- 1.5, $14.80
gpt-5.2: 93.2 +/- 1.3, $16.10
gpt-5.4: 94.2 +/- 1.2, $18.50
kimi-k3@deepinfra: 84.4 +/- 1.9, $18.53
claude-sonnet-4-5: 97.7 +/- 0.4, $19.50
claude-sonnet-4-6: 87.0 +/- 1.7, $19.50
kimi-k3: 84.4 +/- 1.9, $19.50
kimi-k3@moonshot: 84.4 +/- 1.9, $19.50
claude-opus-5-5: 97.1 +/- 0.9, $26.00
gpt-5.6-sol: 96.2 +/- 1.0, $26.00
claude-opus-5: 95.7 +/- 1.0, $32.50
claude-opus-4-8: 94.3 +/- 1.2, $32.50
claude-opus-4-7: 92.9 +/- 1.3, $32.50
claude-opus-4-5: 90.4 +/- 1.5, $32.50
claude-opus-4-6: 89.3 +/- 1.6, $32.50
gpt-5.5: 95.9 +/- 1.0, $37.00
gpt-4-turbo: 46.7 +/- 1.1, $47.00
claude-fable-5-1: 97.0 +/- 0.9, $65.00
gpt-6-astra: 96.8 +/- 0.9, $65.00
claude-fable-5: 96.0 +/- 1.0, $65.00
o1: 81.7 +/- 1.1, $84.00
gpt-4: 23.0 +/- 0.9, $114
deepseek-v4-flash-vision-exp gpt-5-mini Best, Balanced, Cheap