Skip to content
Agents tracked: 492 Releases (7d): 545 Agent status: 1 with issuesSince last limit reset: Claude 9d · Codex 7d For AI agents: Markdown Updated Nov 20, 2026

Groq (LLM inference)

by Groq

Fast answers from open AI models.

Model hosting and inference

WebsitePricingGitHub

1. Price

No free tier on the pricing page. As reported on the official pricing page, checked Oct 22, 2026.

console.groq.com/docs/models

Checked against the maker's official pages, pricing on Oct 22, 2026. Tap a verdict to see its source.How we checkedReport an error

What Groq (LLM inference) does

Groq is an online service that runs open AI language models, such as OpenAI's GPT-OSS, and returns their answers very quickly. Your app sends a question or a task and gets the text back, paying per amount of text. It works with the same code as the OpenAI service once you change the address and the key.

  • Groq (LLM inference) is one of 41 APIs in the Model hosting and inference category on AgentGid.
  • It is not among the 13 that can be started for free.
  • 6 of them ship an official MCP server; Groq (LLM inference) does not.
  • Its official SDK is the 4th most downloaded in the category (4.9M a week).

What people build with it

A chat assistant in your app

Answer your users' questions inside your app or site, with replies that show up word by word while they are written.

Text features in a Next.js app

Write product descriptions, summaries or recipes from a short prompt inside a JavaScript web app built with the Vercel AI toolkit.

Sort and tag incoming emails

Turn free text, like support emails or form answers, into fixed fields your app can store, such as category and urgency.

Answers with today's news

Let the model look things up on the web before it answers, for questions about recent events.

Voice notes into text

Turn recorded voice messages or calls into text with OpenAI's Whisper model, hosted by Groq and billed per hour of audio.

Big jobs overnight at half price

Send thousands of requests at once, such as tagging a whole product catalog, and collect the results within a day or a week.

Groq (LLM inference) popularity

5M
SDK downloads a week
+0.3% on the 4-week average
269
GitHub stars, groq/groq-typescript
SDK downloads per weeknpm groq-sdk, pypi groq 4,951,614

SDK downloads from npm and PyPI, weekly. Numbers as of Nov 17, 2026.

Pricing

Free tier

None

Cheapest plan

No monthly plan

Usage price

gpt-oss-120b $0.15 in / $0.60 out per 1M tokens; gpt-oss-20b $0.075 / $0.30

What one action costs

ActionPrice
1M input tokens, openai/gpt-oss-120b$0.15
1M output tokens, openai/gpt-oss-120b$0.60
1M input tokens, openai/gpt-oss-20b$0.075
1 hour of audio transcribed, whisper-large-v3-turbo$0.04
1 hour of audio transcribed, whisper-large-v3$0.111

As reported on the official pricing page, checked Oct 22, 2026. Prices change: confirm before you buy. console.groq.com/docs/models

Limits

Base limits (table on the rate limits page)

openai/gpt-oss-120b: 30 requests a minute, 1,000 a day, 8,000 tokens a minute, 200,000 tokens a day

Developer

openai/gpt-oss-120b: 1,000 requests and 250,000 tokens a minute

All plans

Limits apply to the whole organization, not to each key or person; cached tokens do not count.

Free / Developer

Audio files up to 25 MB on the free tier and 100 MB on the dev tier

From the official docs. console.groq.com/docs/rate-limits.md

Key and access

How it signs in

API key in the header Authorization: Bearer <key>

Environment variable
GROQ_API_KEY
Safe in browser code
No, keep it on your server
Card at sign-up

No

Steps only a person can do

  • Create a Groq account and an API key. Put it in .env as GROQ_API_KEY (kept out of git) and read it on the server.
  • For higher limits, batch jobs and spend limits, upgrade to the Developer tier with a payment method. No code change; the same key gets the higher limits.
  • Optionally turn on Zero Data Retention for the organization. Do not use the batch endpoints, which are switched off under ZDR.

Commercial use and your data

Groq is a paid online service with a free tier, and you can use it in a commercial product. You own your inputs and outputs; each model also has its own maker's terms you must follow.

In a paid app
With conditions

Allowed; follow Groq's acceptable use policy and the terms of each model's maker.

“Customer and its Authorized Users will comply with the applicable AI Model Terms for each AI Model Service that is used under this Agreement.” source

inside-app-bundle
Yes

The official groq and groq-sdk packages are Apache-2.0; the key itself must stay on your server.

“"license":"Apache-2.0"” source

client-work
Yes

The customer keeps the rights in inputs and outputs; use a key in the client's own Groq organization.

“As between the parties, Customer retains all Intellectual Property Rights in Customer Data (including in Inputs and Outputs), any Customer Application, Customer Training Data, and Customer supplied AI Model Service.” source

To train your own AI
With conditions

Groq does not train on your data; whether you may train on outputs depends on each model's own terms.

“Customer and its Authorized Users will comply with the applicable AI Model Terms for each AI Model Service that is used under this Agreement.” source

Trains on what you send

No

“For clarity, Groq is not permitted to use Inputs or Outputs for training or fine-tuning any AI Model Services or other models, unless explicitly granted permission or instructed by Customer.” source

Keeps your data

Inputs and outputs are not kept by default; they may be logged for up to 30 days to fix failures or check abuse, and batch files are kept up to 30 days.

“By default, Groq does not retain customer data for inference requests.” source

Where data is stored

Stored customer data is kept in Google Cloud buckets in the United States.

“All customer data is retained in Google Cloud Platform (GCP) buckets located in the United States.” source

Errors to handle

401 Unauthorized

The key is missing or not valid.

Check that GROQ_API_KEY is set on the server and that the key was not deleted in the console.

400 Bad Request (also blocked_api_access when a spend limit is reached)

The request has a wrong field, an OpenAI option Groq does not support, or the organization hit its spend limit.

Remove unsupported fields such as logprobs or logit_bias; for blocked_api_access raise or remove the spend limit.

429 Too Many Requests

A rate limit for requests or tokens was reached.

Wait for the seconds in the retry-after header, then retry with backoff; upgrade for higher limits.

498 Flex Tier Capacity Exceeded

The cheaper flex tier is full at the moment.

Retry later or send the request on the standard tier.

500 / 502 / 503

A problem on Groq's side; these requests are not charged.

Retry later with backoff.

Use it from an agent

Our agent check

2 of 6: llms.txt no, OpenAPI no, SDK yes, MCP no, status page no, AI assistants allowed yes

Checked Nov 13, 2026.

MCP server

None found

npm
npm install groq-sdk
PyPI
pip install groq

Alternatives

Groq (LLM inference) This page

from $0.075No free tier

OpenAI API

Pick the OpenAI API when you want OpenAI's own GPT models with vision, tool calling and agents, billed per 1M tokens.

from $0.1No free tier

Gemini API

Pick the Gemini API when you need image, audio or video input as well as text, with a free tier.

Free planFree tier

All model hosting and inference APIs

Get started

Tell your AI assistant

“Add Groq to my app on the server: use the groq package (or the OpenAI client with base_url https://api.groq.com/openai/v1) with GROQ_API_KEY from .env, model openai/gpt-oss-20b, retry 429 and 5xx with backoff, and never put the key in browser code.”

  1. Create an API key in the Groq console and keep it in your environment as GROQ_API_KEY.

    export GROQ_API_KEY=<your-api-key-here>

    From console.groq.com/docs/quickstart.md

  2. Install the official Python package (or keep using the OpenAI package with Groq's address).

    pip install groq

    From console.groq.com/docs/quickstart.md

  3. To reuse OpenAI code, change only the base address and the key.

    client = openai.OpenAI(
        base_url="https://api.groq.com/openai/v1",
        api_key=os.environ.get("GROQ_API_KEY")
    )

    From console.groq.com/docs/openai.md

First request

Trimmed; the full example reply in the docs also lists the settings used. The docs' example reply names llama-3.3-70b-versatile as the model although the request asks for openai/gpt-oss-120b.

curl https://api.groq.com/openai/v1/responses -s \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $GROQ_API_KEY" \
-d '{
  "model": "openai/gpt-oss-120b",
  "input": "Tell me a three sentence bedtime story about a unicorn."
}'

Mistakes to avoid

Copying model llama-3.3-70b-versatile from the quickstart into a new app.
Use a production model listed with a price, such as openai/gpt-oss-20b or openai/gpt-oss-120b.

The models page now lists Llama 3.3 70B as Enterprise with Contact Sales pricing.

Setting dangerouslyAllowBrowser to true and calling Groq from the web page.
Call Groq from a server route and keep GROQ_API_KEY on the server.

Anyone who opens the page could copy the key and spend your money.

Sending OpenAI options logprobs, logit_bias or n greater than 1.
Leave them out; Groq answers them with a 400 error.

These OpenAI fields are not supported on Groq.

Asking for strict JSON output and web search in the same request.
Run browser search first, then a second request with structured outputs on the answer.

Browser search does not work together with structured outputs.

Using whisper-large-v3-turbo to translate audio into English.
Use whisper-large-v3 for translation; turbo only transcribes.

The speech-to-text table marks translation as not supported for turbo.

Questions

Is there a free tier?

Yes. The Free tier needs no payment method; you add one only to move to the pay-as-you-go Developer tier.

Can I keep my OpenAI code?

Mostly. Point the OpenAI client at https://api.groq.com/openai/v1 with a Groq key; a few options like logprobs are not supported.

Does Groq train on my prompts?

No. Its agreement says inputs and outputs are not used for training unless you allow it.

Can I cap my bill?

Yes, on paid plans. A spend limit blocks all keys of the organization when the monthly cap is reached.

Am I charged for failed requests?

Not for errors on Groq's side (500, 502, 503).