Llama 3.1 8B Instruct
meta-llama/llama-3.1-8b-instruct
Compare with Llama 3.1 70B Instruct, Llama 3.2 3B Instruct, Llama 3.2 1B Instruct
CompareLlama 3.1 8B Instruct
| State | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Healthy | 100% | 99.91% | 99.90% | 99.90% | 499ms | 2.55s | 17 tps | 32K | $0.152 / $0.287 | |
| Healthy | 100% | 100% | 99.99% | 99.99% | 177ms | 507ms | 92 tps | 128K | $0.220 / $0.220 | |
| Healthy | 99.95% | 99.88% | 99.36% | 99.36% | 487ms | 3.08s | 25 tps | 131K | $0.020 / $0.040 | |
Groqgroq | Healthy | 99.84% | 99.93% | 99.90% | 99.90% | 246ms | 1.02s | 116 tps | 131K | $0.050 / $0.080 |
| Healthy | 99.59% | 99.78% | 99.77% | 99.77% | 491ms | 1.90s | 52 tps | 16K | $0.020 / $0.050 |
- Cloudflare100%Healthy24h 99.91%
- 7d
- 99.90%
- 30d
- 99.90%
- p50
- 499ms
- p99
- 2.55s
- Throughput
- 17 tps
- Context
- 32K
- $/Mtok
- $0.152 / $0.287
cloudflare/fp8fp8 - CoreWeave100%Healthy24h 100%
- 7d
- 99.99%
- 30d
- 99.99%
- p50
- 177ms
- p99
- 507ms
- Throughput
- 92 tps
- Context
- 128K
- $/Mtok
- $0.220 / $0.220
coreweave/bf16bf16 - DeepInfra99.95%Healthy24h 99.88%
- 7d
- 99.36%
- 30d
- 99.36%
- p50
- 487ms
- p99
- 3.08s
- Throughput
- 25 tps
- Context
- 131K
- $/Mtok
- $0.020 / $0.040
deepinfra/fp8fp8 - Groq99.84%Healthy24h 99.93%
- 7d
- 99.90%
- 30d
- 99.90%
- p50
- 246ms
- p99
- 1.02s
- Throughput
- 116 tps
- Context
- 131K
- $/Mtok
- $0.050 / $0.080
groq - Novita99.59%Healthy24h 99.78%
- 7d
- 99.77%
- 30d
- 99.77%
- p50
- 491ms
- p99
- 1.90s
- Throughput
- 52 tps
- Context
- 16K
- $/Mtok
- $0.020 / $0.050
novita/fp8fp8
Measured 2m ago. Latency and throughput are OpenRouter's rolling 30-minute windows.
History
Availability
Axis starts at 90% so small differences stay visible; it expands to 0 when an endpoint falls below.
Time to first token
Logarithmic axis — the fleet spans two orders of magnitude. P50 is OpenRouter's rolling 30-minute window.
Throughput
Median tokens per second.