Llama 3.1 8B Instruct: providers compared
Every endpoint serving this model, side by side. ◆ marks the best value in each column — per model, never fleet-wide.
Current state
| State | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Healthy | 100%◆ (best) | 100% | 100%◆ (best) | 100%◆ (best) | 194ms◆ (best) | 1.11s◆ (best) | 92 tps | 128K | $0.220 / $0.220 | |
| Healthy | 100%◆ (best) | 99.80% | 99.78% | 99.78% | 493ms | 1.43s | 49 tps | 16K | $0.020 / $0.050◆ (best) | |
Groqgroq | Healthy | 99.94% | 99.93% | 99.90% | 99.90% | 259ms | 1.40s | 101 tps◆ (best) | 131K | $0.050 / $0.080 |
| Healthy | 99.92% | 99.91% | 99.41% | 99.41% | 699ms | 14.1s | 15 tps | 131K | $0.020 / $0.040◆ (best) | |
| Healthy | 99.63% | 99.81% | 99.89% | 99.89% | 559ms | 3.09s | 15 tps | 32K | $0.152 / $0.287 |
- CoreWeave100%◆ (best)Healthy24h 100%
- 7d
- 100%◆ (best)
- 30d
- 100%◆ (best)
- p50
- 194ms◆ (best)
- p99
- 1.11s◆ (best)
- Throughput
- 92 tps
- Context
- 128K
- $/Mtok
- $0.220 / $0.220
coreweave/bf16bf16 - Novita100%◆ (best)Healthy24h 99.80%
- 7d
- 99.78%
- 30d
- 99.78%
- p50
- 493ms
- p99
- 1.43s
- Throughput
- 49 tps
- Context
- 16K
- $/Mtok
- $0.020 / $0.050◆ (best)
novita/fp8fp8 - Groq99.94%Healthy24h 99.93%
- 7d
- 99.90%
- 30d
- 99.90%
- p50
- 259ms
- p99
- 1.40s
- Throughput
- 101 tps◆ (best)
- Context
- 131K
- $/Mtok
- $0.050 / $0.080
groq - DeepInfra99.92%Healthy24h 99.91%
- 7d
- 99.41%
- 30d
- 99.41%
- p50
- 699ms
- p99
- 14.1s
- Throughput
- 15 tps
- Context
- 131K
- $/Mtok
- $0.020 / $0.040◆ (best)
deepinfra/fp8fp8 - Cloudflare99.63%Healthy24h 99.81%
- 7d
- 99.89%
- 30d
- 99.89%
- p50
- 559ms
- p99
- 3.09s
- Throughput
- 15 tps
- Context
- 32K
- $/Mtok
- $0.152 / $0.287
cloudflare/fp8fp8
Measured 5m ago. Latency and throughput are OpenRouter's rolling 30-minute windows. ◆ marks the best value in each column among these endpoints.
Price vs latency
One dot per endpoint at the latest round · further left = faster, lower = cheaper · hover a dot for its tag
History
Availability
Axis starts at 90% so small differences stay visible; it expands to 0 when an endpoint falls below.
Time to first token
Logarithmic axis — the fleet spans two orders of magnitude. P50 is OpenRouter's rolling 30-minute window.
Throughput
Median tokens per second.