Llama 3.1 8B Instruct

131K ctxText Tools

Released Jul 23, 2024 · 73B tokens this week · 5 providers

Overview

A small, extremely cheap open model that still handles chat, summarising and simple extraction well. Common as a draft model for speculative decoding and as a cheap classifier. Runs comfortably on a single consumer GPU if you self-host.

translationroleplaymarketing

Providers

Reference data — not yet measured from live traffic

PROVIDER		MAX OUT		OUTPUT /M				QUANT
DeepInfraCheapest	131K	16K	$0.030	$0.050	260ms	220	98.94%	fp8
Novita AI	131K	16K	$0.032	$0.053	300ms	185	98.40%	fp8
Together AI	131K	16K	$0.034	$0.056	195ms	260	99.44%	Full
Groq	33K	16K	$0.035	$0.059	209ms	1250	99.85%	Full
Cerebras	33K	16K	$0.036	$0.060	170ms	1900	99.70%	Full

Latency p50 reflects a provider’s API endpoint responsiveness — the round-trip to its API, not per-token inference time. These figures, with throughput and uptime, are reference data until the gateway aggregates its own traffic.

Price comparison

Blended $ per 1M tokens

Llama 3.1 8B Instruct$0.045

DeepSeek V3.2$0.37

Gemini 3 Flash$1.95

GPT-5.2$10.94

Claude Sonnet 4.5$12.00

Blended rate per 1M tokens, weighted one part prompt to three parts completion — roughly the shape of a chat workload. Your mix will move the number.

Call it

OpenAI-compatible — swap the base URL and go

curl https://model.cards/api/v1/chat/completions \
  -H "Authorization: Bearer $MODELCARDS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/llama-3.1-8b",
    "messages": [
      { "role": "user", "content": "Summarize the tradeoffs of speculative decoding." }
    ]
  }'