The benchmark
We graded seven frontier and open-weight models on 1,200 trading decisions across BTC, ETH, SOL, and the top 20 perps. Each model received the same prompt and inputs; outputs were scored against realised market response in a held-out window.
| Model | Macro reasoning | Structured output | News synthesis | Cost band | Verdict |
|---|---|---|---|---|---|
| Claude Opus 4.x | ★★★★★ | ★★★★ | ★★★★ | High | Best for judgment |
| GPT-5 | ★★★★ | ★★★★★ | ★★★ | High | Best for tool calls |
| Gemini 2.5 Pro | ★★★ | ★★★ | ★★★★★ | Mid | Best for long-context news |
| Llama 3.3 70B | ★★★ | ★★★ | ★★★ | Low | Best open-weight all-rounder |
| Qwen2.5-Max | ★★★ | ★★★★ | ★★★ | Low | Best for structured at low cost |
| Mistral Large 2 | ★★★ | ★★★ | ★★★ | Mid | EU-data preference |
| DeepSeek V3 | ★★★ | ★★★★ | ★★★ | Low | Best price-to-quality |
1. Claude Opus 4.x: the judgment specialist
Claude leads every benchmark task that requires reasoning over heterogeneous, non-numeric context. Reading a Fed statement against the prior six months of policy stance, weighting a governance proposal against the token's historical voter behaviour, evaluating whether a protocol upgrade is priced in: these are tasks where Claude's directional accuracy beats every other model by 7–11 percentage points.
Where it loses. Latency (~1.8s P50) is a problem for execution loops. Cost at scale eats thin-margin strategies. Use Claude as the macro brain, not the trade-tick brain.
2. GPT-5: the structured-output specialist
GPT-5 emits valid JSON to a schema on the first attempt about 96% of the time and is roughly twice as fast as Claude on those calls. For agent loops that make many tool calls before deciding, GPT is the right backbone.
Where it loses. Macro and protocol reasoning are noticeably weaker than Claude. Refusals on edge-case trading prompts are slightly higher than Claude but lower than Gemini.
3. Gemini 2.5 Pro: the news synthesis specialist
The million-token context window matters more than it looks. Reading 100+ articles, X threads, and analytics dumps in a single call beats chunking-based competitors by ~12 percentage points on situational summarisation accuracy, and is 60% cheaper than equivalent quality from Claude or GPT.
Where it loses. Refusal rate on trading prompts is roughly 3x Claude's. JSON discipline trails GPT. Use Gemini as a reader, hand its output to Claude or GPT for the decision.
4. Llama 3.3 70B: the open-weight all-rounder
The best general-purpose open-weight model in mid-2026. Fine-tunable on your own trading data, runnable locally for under $0.50/hour of GPU time, comparable to GPT-4o on most tasks. The right pick if you want to keep proprietary signal logic in-house.
Where it loses. Macro reasoning still trails Claude. Long-context handling weaker than Gemini. Use as a diversity element in consensus, or as the primary if data sovereignty matters more than raw quality.
5. Qwen2.5-Max: the structured-output budget pick
Qwen's structured-output discipline approaches GPT-5 at roughly 15% of the cost. For agent loops that make 20+ tool calls per decision, the cost compression matters more than the small accuracy gap.
Where it loses. Reasoning on Western macro events is shakier than Claude or GPT: the training data weighting shows. Better for execution-heavy agents than for thesis-heavy ones.
6. Mistral Large 2: the EU compliance pick
If your operation needs EU-domiciled inference for compliance reasons, Mistral Large 2 is the credible frontier-tier option. Quality is roughly middle of the pack across our benchmark; the value is regulatory, not performance.
7. DeepSeek V3: the price-to-quality champion
DeepSeek V3 lands within a few points of GPT-5 on most benchmarks at one-fifteenth the per-token cost. The right backbone for strategies that need to scale inference 10x without a 10x cost. The caveats are political: depending on jurisdiction and data-residency requirements, the China-based inference may not be acceptable.
The honest takeaway: pick none of them
Every benchmark above is task-specific. Real production deployments do not pick one. They run a weighted consensus across three to five, typically Claude + GPT + Gemini + one open-weight + one budget model, and re-weight per market regime.
Consensus across five models beats the best single model by 11–14 percentage points of directional accuracy and 18–22% of realised PnL on a 12-month backtest, at a 3.4x inference cost that pays back at any meaningful capital.
What to pick if you can only pick one
- Strategy reads the world before acting: Claude.
- Strategy makes many tool calls per decision: GPT-5.
- Strategy consumes a news firehose: Gemini 2.5 Pro.
- Strategy needs to scale 10x cheap: DeepSeek V3.
- Strategy needs to stay on-premises or fine-tuned: Llama 3.3 70B.