NVIDIA

Llama 3.1 Nemotron 70B

131K ctxText

Released Oct 15, 2024 · 17B tokens this week · 4 providers

Overview

NVIDIA's alignment-tuned Llama 3.1 70B, reworked with reward-model feedback for more helpful answers. Effectively a drop-in upgrade for teams already serving Llama 70B. Same architecture, so existing serving stacks need no changes.

roleplayacademiatranslation

Providers

Reference data — not yet measured from live traffic

PROVIDER		MAX OUT		OUTPUT /M				QUANT
DeepInfraCheapest	131K	33K	$0.12	$0.30	480ms	105	98.60%	fp8
Novita AI	131K	33K	$0.13	$0.33	540ms	88	98.30%	fp8
Lambda	131K	33K	$0.14	$0.34	620ms	70	98.05%	Full
Together AI	131K	33K	$0.14	$0.35	195ms	128	99.30%	Full

Latency p50 reflects a provider’s API endpoint responsiveness — the round-trip to its API, not per-token inference time. These figures, with throughput and uptime, are reference data until the gateway aggregates its own traffic.

Price comparison

Blended $ per 1M tokens

Llama 3.1 Nemotron 70B$0.26

DeepSeek V3.2$0.37

Gemini 3 Flash$1.95

GPT-5.2$10.94

Claude Sonnet 4.5$12.00

Blended rate per 1M tokens, weighted one part prompt to three parts completion — roughly the shape of a chat workload. Your mix will move the number.

Call it

OpenAI-compatible — swap the base URL and go

curl https://model.cards/api/v1/chat/completions \
  -H "Authorization: Bearer $MODELCARDS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nvidia/llama-3.1-nemotron-70b",
    "messages": [
      { "role": "user", "content": "Summarize the tradeoffs of speculative decoding." }
    ]
  }'