TokenMix Research Lab · 2026-06-04

Groq API Access 2026: Free Tier, Rate Limits, Key Setup

Last Updated: 2026-06-04 Author: TokenMix Research Lab Data verified: 2026-06-04 - Groq quickstart, OpenAI compatibility, rate limits, pricing, supported models, Batch API, Flex Processing, error docs, and spend-limit docs

Groq API access is easy: create a GroqCloud key, set GROQ_API_KEY, and call the OpenAI-compatible endpoint. The hard part is free-tier math, because model-specific RPM, RPD, TPM, and TPD caps decide whether your workload actually fits.

Groq publishes concrete free limits by model, not one universal free bucket. As of June 4, 2026, llama-3.3-70b-versatile is listed at 30 RPM, 1K RPD, 12K TPM, and 100K TPD on the Free Plan, while openai/gpt-oss-120b is listed at 30 RPM, 1K RPD, 8K TPM, and 200K TPD (Groq rate limits). The same GPT OSS 120B model is priced at $0.15 input and $0.60 output per 1M tokens on Groq's official pricing page, with prompt cache hits listed at $0.075 input (Groq pricing). Groq also supports OpenAI-compatible calls through https://api.groq.com/openai/v1, but the compatibility is "mostly compatible," not feature-complete (OpenAI compatibility). Batch Processing gives a 50% cost discount for async jobs, while Flex Processing gives paid customers higher rate limits but can fail fast with status 498 when flex capacity is unavailable (Batch API, Flex Processing).

Table of Contents

• Quick Verdict
• Access Checklist
• API Key Setup
• Free Tier Rate Limits
• Free Tier Cost Math
• Official Pricing
• Workload Cost Projections
• Error Codes and Fixes
• Batch Flex and Caching
• Use Case Matrix
• Risks and Caveats
• Final Recommendation
• FAQ
• Sources
• Related Articles

Quick Verdict

Access Checklist

The correct starting point is simple. The risky part is assuming the free tier means "free production inference."

If you are comparing Groq against broader free API options, the clean cluster read is Free LLM API 2026. For production routing, pair this with AI API Gateway 2026, because free-tier access without fallback is fragile.

API Key Setup

Groq's native SDK path is the fastest way to start. The OpenAI-compatible path is better if you already have a client wrapper, gateway, eval harness, or routing layer.

Native Python SDK:

import os
from groq import Groq

client = Groq(api_key=os.environ["GROQ_API_KEY"])

completion = client.chat.completions.create(
    model="openai/gpt-oss-120b",
    messages=[
        {"role": "system", "content": "Answer in one paragraph."},
        {"role": "user", "content": "Explain Groq API rate limits in plain English."},
    ],
)

print(completion.choices[0].message.content)

OpenAI-compatible Python client:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["GROQ_API_KEY"],
    base_url="https://api.groq.com/openai/v1",
)

response = client.chat.completions.create(
    model="openai/gpt-oss-120b",
    messages=[{"role": "user", "content": "Give me a 5-item API launch checklist."}],
)

print(response.choices[0].message.content)

cURL:

curl https://api.groq.com/openai/v1/chat/completions \
  -H "Authorization: Bearer $GROQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-oss-120b",
    "messages": [
      {"role": "user", "content": "What is the difference between RPM and TPM?"}
    ]
  }'

The practical migration note: if your app already points to OpenAI, changing the base_url may be enough for basic chat completions. It is not enough for every OpenAI feature. Groq documents unsupported fields such as logprobs, logit_bias, top_logprobs, and messages[].name, and says n must equal 1 if supplied (OpenAI compatibility).

Free Tier Rate Limits

Groq's free plan is useful, but the limit that bites first depends on your average request size.

Groq also publishes response headers for limits, remaining quota, reset windows, and retry-after. The important confirmed detail: x-ratelimit-limit-requests refers to RPD, while x-ratelimit-limit-tokens refers to TPM in the documented header examples (Groq rate limits). Treat headers as the runtime source of truth.

Free Tier Cost Math

Free-tier planning should start with tokens per call, not requests per day. A 1K RPD number can be irrelevant if TPD is lower.

Three concrete takeaways:

1. openai/gpt-oss-120b at 2,000 total tokens per call is not a 1,000-call/day free backend. It is about 100 calls/day before 200K TPD.
2. llama-3.3-70b-versatile has a stronger 12K TPM burst window than GPT OSS 120B, but its 100K TPD means the day can end faster for medium chat.
3. qwen/qwen3-32b has 60 RPM but only 6K TPM, so twelve 500-token calls can exhaust one minute of token capacity even though request capacity looks larger.

This is the same hidden math developers miss when they compare free providers. A free model with impressive RPM can still lose to a lower-RPM provider if the token/day cap is tight. The broader provider comparison sits in Cheapest AI API Providers 2026.

Official Pricing

Groq pricing is pay-per-token for listed LLMs, with separate units for speech and built-in tools. The table below uses only official published prices visible on Groq pricing or supported-model pages as of this verification date.

The speed mismatch between Groq's pricing and model pages is not a contradiction worth over-reading. It is a reminder to label speed as page-reported product data, not as a guaranteed SLA. Groq's enterprise Performance Tier separately describes a 99.9% availability SLA and latency guarantee aligned to enterprise agreements (Performance Tier).

Workload Cost Projections

The paid math is straightforward. Multiply input tokens by input price and output tokens by output price. The tricky part is choosing the model that fits the job.

Cost example 1: 50 support tickets/day, 30 days, 2K input and 500 output tokens each equals 3M input and 0.75M output tokens/month. On GPT OSS 20B, that is 3M x $0.075 + 0.75M x $0.30 = $0.45/month. The free tier may cover it if daily TPD and burst caps fit.

Cost example 2: 2,000 RAG calls/day, 30 days, 1,500 input and 500 output tokens each equals 90M input and 30M output tokens/month. On GPT OSS 120B, that is 90M x $0.15 + 30M x $0.60 = $31.50/month. Batch would cut eligible async work to $15.75, but live chat needs synchronous routing.

Cost example 3: 100 developer-agent runs/day, 30 days, 20K input and 4K output tokens each equals 60M input and 12M output tokens/month. On Llama 3.3 70B, that is 60M x $0.59 + 12M x $0.79 = $44.88/month. On GPT OSS 120B, the same token shape is $16.20/month. Whether quality is good enough is task-specific; the price gap is confirmed.

For multi-provider fallback and budget caps, compare this with OpenRouter Alternatives 2026. The cost math changes once a gateway adds markup, routing fees, or provider-specific billing rules.

Error Codes and Fixes

The most common Groq setup failure is not model quality. It is a small auth, payload, model, rate-limit, or tier mistake.

A minimal 429 handler should inspect headers, not blindly sleep:

import random
import time

def groq_retry_sleep(response_headers, attempt):
    retry_after = response_headers.get("retry-after")
    if retry_after:
        return float(retry_after)

    base = min(2 ** attempt, 30)
    jitter = random.uniform(0, 0.5 * base)
    return base + jitter

def should_retry(status_code):
    return status_code in {429, 498, 500, 502, 503, 504}

That function is deliberately conservative. For a real launch, track RPM, RPD, TPM, and TPD per model. A request can be legal by RPM and still illegal by TPM.

Batch Flex and Caching

Groq now has three different cost/capacity levers that developers often mix up.

The discount rule matters: Batch and prompt caching do not stack. If a 100M input-token GPT OSS 120B batch is eligible for Batch, the input cost is 100M x $0.15 x 50% = $7.50. It is not 100M x $0.075 x 50% = $3.75. Groq says batch tokens are billed at the 50% batch rate regardless of cache status (Batch API, Prompt caching).

Use Case Matrix

Risks and Caveats

Final Recommendation

Use Groq free access for prototypes, evals, and fast short calls. For production, start paid with spend limits, token-aware throttling, Batch for async jobs, and fallback routing. The best default test path is GPT OSS 120B; the best cheap paid path is workload-dependent.

FAQ

How do I get Groq API access?

Create a GroqCloud account, generate an API key, and set it as GROQ_API_KEY. Then use either the Groq SDK or an OpenAI-compatible client pointed at https://api.groq.com/openai/v1.

Is Groq API free in 2026?

Yes, Groq has a Free Plan, but it is rate limited by model. Treat it as a prototype and evaluation lane, not a guaranteed production backend.

What are Groq free tier rate limits?

They vary by model. For example, Groq lists llama-3.3-70b-versatile at 30 RPM, 1K RPD, 12K TPM, and 100K TPD, while openai/gpt-oss-120b is 30 RPM, 1K RPD, 8K TPM, and 200K TPD.

Which Groq model should I start with?

Start with openai/gpt-oss-120b if you want a strong general test model. Start with llama-3.1-8b-instant if the task is simple extraction and cost matters more than reasoning depth.

Is Groq fully OpenAI-compatible?

No. Groq is mostly OpenAI-compatible, but not every OpenAI field is supported. Groq documents unsupported fields such as logprobs, logit_bias, top_logprobs, and messages[].name.

How do I fix Groq 429 errors?

Throttle requests, respect retry-after, and track both request and token buckets. A 429 can be caused by RPM, RPD, TPM, or TPD depending on model and traffic shape.

Does Groq Batch API reduce cost?

Yes. Groq says Batch Processing has 50% lower cost than synchronous APIs. It is best for offline jobs because the processing window is 24 hours to 7 days.

Can I use Groq for production?

Yes, but not casually on the Free Plan. Use production models, paid limits, spend caps, monitoring, retries, and fallback routing before sending real user traffic.

Sources

• Groq Quickstart - official API key and SDK setup path
• Groq OpenAI Compatibility - official base URL and unsupported OpenAI fields
• Groq Rate Limits - official RPM, RPD, TPM, TPD, headers, and 429 behavior
• Groq Pricing - official model pricing, prompt caching prices, speech pricing, and tool prices
• Groq Supported Models - official production and preview model table
• Groq Batch API - official async batch API and 50% cost discount
• Groq Flex Processing - official paid Flex tier behavior and 498 capacity_exceeded
• Groq Error Codes - official HTTP status code reference
• Groq Prompt Caching - official cache behavior and non-stacking batch rule
• Groq Spend Limits - official monthly spend cap behavior and tracking delay
• Groq Performance Tier - official enterprise SLA and provisioned-capacity framing

Related Articles

• Groq API Pricing 2026: Free Tier, 315 TPS, $0.05/M Paid Models
• Free LLM API 2026: 15 Limits, No-Card Picks, Real Costs
• Cheapest AI API Providers 2026: Every Provider Ranked by $/M
• 8 OpenRouter Alternatives 2026: Free or Below-Market Pricing
• AI API Gateway 2026: Routing, Fallbacks, Observability, and Cost Control