Llama 3.1 8B Instruct: providers compared
Every endpoint serving this model, side by side. ◆ marks the best value in each column — per model, never fleet-wide.
Current state
| State | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Healthy | 100%◆ (best) | 97.61% | 99.21% | 98.85% | 518ms | 1.74s | 18 tps | 32K | $0.152 / $0.287 | |
| Healthy | 100%◆ (best) | 100% | 100%◆ (best) | 99.97%◆ (best) | 252ms◆ (best) | 556ms◆ (best) | 127 tps | 131K | $0.220 / $0.220 | |
| Healthy | 100%◆ (best) | 99.60% | 99.16% | 98.78% | 536ms | 2.55s | 99 tps | 16K | $0.020 / $0.050◆ (best) | |
Groqgroq | Healthy | 99.86% | 99.70% | 99.73% | 99.64% | 263ms | 5.02s | 264 tps◆ (best) | 131K | $0.050 / $0.080 |
| Healthy | 95.64% | 99.43% | 99.16% | 99.21% | 796ms | 4.69s | 13 tps | 131K | $0.020 / $0.040◆ (best) |
- Cloudflare100%◆ (best)Healthy24h 97.61%
- 7d
- 99.21%
- 30d
- 98.85%
- p50
- 518ms
- p99
- 1.74s
- Throughput
- 18 tps
- Context
- 32K
- $/Mtok
- $0.152 / $0.287
cloudflare/fp8fp8 - CoreWeave100%◆ (best)Healthy24h 100%
- 7d
- 100%◆ (best)
- 30d
- 99.97%◆ (best)
- p50
- 252ms◆ (best)
- p99
- 556ms◆ (best)
- Throughput
- 127 tps
- Context
- 131K
- $/Mtok
- $0.220 / $0.220
coreweave/bf16bf16 - Novita100%◆ (best)Healthy24h 99.60%
- 7d
- 99.16%
- 30d
- 98.78%
- p50
- 536ms
- p99
- 2.55s
- Throughput
- 99 tps
- Context
- 16K
- $/Mtok
- $0.020 / $0.050◆ (best)
novita/fp8fp8 - Groq99.86%Healthy24h 99.70%
- 7d
- 99.73%
- 30d
- 99.64%
- p50
- 263ms
- p99
- 5.02s
- Throughput
- 264 tps◆ (best)
- Context
- 131K
- $/Mtok
- $0.050 / $0.080
groq - DeepInfra95.64%Healthy24h 99.43%
- 7d
- 99.16%
- 30d
- 99.21%
- p50
- 796ms
- p99
- 4.69s
- Throughput
- 13 tps
- Context
- 131K
- $/Mtok
- $0.020 / $0.040◆ (best)
deepinfra/fp8fp8
Measured 2m ago. Latency and throughput are OpenRouter's rolling 30-minute windows. ◆ marks the best value in each column among these endpoints.
Price vs latency
One dot per endpoint at the latest round · further left = faster, lower = cheaper · hover a dot for its tag
History
Availability
Axis starts at 90% so small differences stay visible; it expands to 0 when an endpoint falls below.
Time to first token
Logarithmic axis — the fleet spans two orders of magnitude. P50 is OpenRouter's rolling 30-minute window, averaged over the hour.
Throughput
Median tokens per second.