Llama 3.1 8B Instruct
meta-llama/llama-3.1-8b-instruct
Compare with Llama 3.1 70B Instruct, Llama 3.2 3B Instruct, Llama 3.2 1B Instruct
CompareLlama 3.1 8B Instruct
| Provider | State | Uptime 5m | 24h | p50 latency | p99 | Throughput | Context | $/Mtok |
|---|---|---|---|---|---|---|---|---|
Cloudflarecloudflare/fp8fp8 | Healthy | 100% | 99.68% | 496ms | 3.02s | 15 tps | 32K | $0.152 / $0.287 |
CoreWeavecoreweave/bf16bf16 | Healthy | 100% | 100% | 145ms | 535ms | 103 tps | 128K | $0.220 / $0.220 |
DeepInfradeepinfra/fp8fp8 | Healthy | 100% | 98.33% | 592ms | 11.7s | 18 tps | 131K | $0.020 / $0.040 |
Groqgroq | Healthy | 99.92% | 99.85% | 197ms | 1.06s | 128 tps | 131K | $0.050 / $0.080 |
Novitanovita/fp8fp8 | Healthy | 99.89% | 99.46% | 496ms | 2.53s | 63 tps | 16K | $0.020 / $0.050 |
- Cloudflare100%Healthy24h 99.68%
- p50
- 496ms
- p99
- 3.02s
- Throughput
- 15 tps
- Context
- 32K
- $/Mtok
- $0.152 / $0.287
cloudflare/fp8fp8 - CoreWeave100%Healthy24h 100%
- p50
- 145ms
- p99
- 535ms
- Throughput
- 103 tps
- Context
- 128K
- $/Mtok
- $0.220 / $0.220
coreweave/bf16bf16 - DeepInfra100%Healthy24h 98.33%
- p50
- 592ms
- p99
- 11.7s
- Throughput
- 18 tps
- Context
- 131K
- $/Mtok
- $0.020 / $0.040
deepinfra/fp8fp8 - Groq99.92%Healthy24h 99.85%
- p50
- 197ms
- p99
- 1.06s
- Throughput
- 128 tps
- Context
- 131K
- $/Mtok
- $0.050 / $0.080
groq - Novita99.89%Healthy24h 99.46%
- p50
- 496ms
- p99
- 2.53s
- Throughput
- 63 tps
- Context
- 16K
- $/Mtok
- $0.020 / $0.050
novita/fp8fp8
Measured 3m ago. Latency and throughput are OpenRouter's rolling 30-minute windows.
History
Availability
Axis starts at 90% so small differences stay visible; it expands to 0 when an endpoint falls below.
Time to first token
Logarithmic axis — the fleet spans two orders of magnitude. p50 and p99 are OpenRouter's rolling 30-minute windows.
Throughput
Median tokens per second.