Groq (LLM inference)
by Groq
Fast answers from open AI models.
1. Price
No free tier on the pricing page. As reported on the official pricing page, checked Oct 22, 2026.
console.groq.com/docs/models2. Commercial use
“Customer and its Authorized Users will comply with the applicable AI Model Terms for each AI Model Service that is used under this Agreement.”
Allowed; follow Groq's acceptable use policy and the terms of each model's maker. Data: No
console.groq.com/docs/legal/services-agreement.md3. Key & sign-up
“Configure your API key as an environment variable.”
API key in the header Authorization: Bearer <key>
console.groq.com/docs/quickstart.mdChecked against the maker's official pages, pricing on Oct 22, 2026. Tap a verdict to see its source.How we checkedReport an error
What Groq (LLM inference) does
Groq is an online service that runs open AI language models, such as OpenAI's GPT-OSS, and returns their answers very quickly. Your app sends a question or a task and gets the text back, paying per amount of text. It works with the same code as the OpenAI service once you change the address and the key.
- Groq (LLM inference) is one of 41 APIs in the Model hosting and inference category on AgentGid.
- It is not among the 13 that can be started for free.
- 6 of them ship an official MCP server; Groq (LLM inference) does not.
- Its official SDK is the 4th most downloaded in the category (4.9M a week).
What people build with it
A chat assistant in your app
Answer your users' questions inside your app or site, with replies that show up word by word while they are written.
Text features in a Next.js app
Write product descriptions, summaries or recipes from a short prompt inside a JavaScript web app built with the Vercel AI toolkit.
Sort and tag incoming emails
Turn free text, like support emails or form answers, into fixed fields your app can store, such as category and urgency.
Answers with today's news
Let the model look things up on the web before it answers, for questions about recent events.
Voice notes into text
Turn recorded voice messages or calls into text with OpenAI's Whisper model, hosted by Groq and billed per hour of audio.
Big jobs overnight at half price
Send thousands of requests at once, such as tagging a whole product catalog, and collect the results within a day or a week.
Groq (LLM inference) popularity
SDK downloads from npm and PyPI, weekly. Numbers as of Nov 17, 2026.
Pricing
None
No monthly plan
gpt-oss-120b $0.15 in / $0.60 out per 1M tokens; gpt-oss-20b $0.075 / $0.30
What one action costs
| Action | Price |
|---|---|
| 1M input tokens, openai/gpt-oss-120b | $0.15 |
| 1M output tokens, openai/gpt-oss-120b | $0.60 |
| 1M input tokens, openai/gpt-oss-20b | $0.075 |
| 1 hour of audio transcribed, whisper-large-v3-turbo | $0.04 |
| 1 hour of audio transcribed, whisper-large-v3 | $0.111 |
As reported on the official pricing page, checked Oct 22, 2026. Prices change: confirm before you buy. console.groq.com/docs/models
Limits
openai/gpt-oss-120b: 30 requests a minute, 1,000 a day, 8,000 tokens a minute, 200,000 tokens a day
openai/gpt-oss-120b: 1,000 requests and 250,000 tokens a minute
Limits apply to the whole organization, not to each key or person; cached tokens do not count.
Audio files up to 25 MB on the free tier and 100 MB on the dev tier
From the official docs. console.groq.com/docs/rate-limits.md
Key and access
API key in the header Authorization: Bearer <key>
GROQ_API_KEYNo
Steps only a person can do
- Create a Groq account and an API key. Put it in .env as GROQ_API_KEY (kept out of git) and read it on the server.
- For higher limits, batch jobs and spend limits, upgrade to the Developer tier with a payment method. No code change; the same key gets the higher limits.
- Optionally turn on Zero Data Retention for the organization. Do not use the batch endpoints, which are switched off under ZDR.
Commercial use and your data
Groq is a paid online service with a free tier, and you can use it in a commercial product. You own your inputs and outputs; each model also has its own maker's terms you must follow.
Allowed; follow Groq's acceptable use policy and the terms of each model's maker.
“Customer and its Authorized Users will comply with the applicable AI Model Terms for each AI Model Service that is used under this Agreement.” source
The official groq and groq-sdk packages are Apache-2.0; the key itself must stay on your server.
“"license":"Apache-2.0"” source
The customer keeps the rights in inputs and outputs; use a key in the client's own Groq organization.
“As between the parties, Customer retains all Intellectual Property Rights in Customer Data (including in Inputs and Outputs), any Customer Application, Customer Training Data, and Customer supplied AI Model Service.” source
Groq does not train on your data; whether you may train on outputs depends on each model's own terms.
“Customer and its Authorized Users will comply with the applicable AI Model Terms for each AI Model Service that is used under this Agreement.” source
No
“For clarity, Groq is not permitted to use Inputs or Outputs for training or fine-tuning any AI Model Services or other models, unless explicitly granted permission or instructed by Customer.” source
Inputs and outputs are not kept by default; they may be logged for up to 30 days to fix failures or check abuse, and batch files are kept up to 30 days.
“By default, Groq does not retain customer data for inference requests.” source
Stored customer data is kept in Google Cloud buckets in the United States.
“All customer data is retained in Google Cloud Platform (GCP) buckets located in the United States.” source
Errors to handle
The key is missing or not valid.
Check that GROQ_API_KEY is set on the server and that the key was not deleted in the console.
The request has a wrong field, an OpenAI option Groq does not support, or the organization hit its spend limit.
Remove unsupported fields such as logprobs or logit_bias; for blocked_api_access raise or remove the spend limit.
A rate limit for requests or tokens was reached.
Wait for the seconds in the retry-after header, then retry with backoff; upgrade for higher limits.
The cheaper flex tier is full at the moment.
Retry later or send the request on the standard tier.
A problem on Groq's side; these requests are not charged.
Retry later with backoff.
Use it from an agent
2 of 6: llms.txt no, OpenAPI no, SDK yes, MCP no, status page no, AI assistants allowed yes
Checked Nov 13, 2026.
None found
npm install groq-sdkpip install groqAlternatives
Groq (LLM inference) This page
Pick the OpenAI API when you want OpenAI's own GPT models with vision, tool calling and agents, billed per 1M tokens.
Pick the Gemini API when you need image, audio or video input as well as text, with a free tier.
Get started
Tell your AI assistant
“Add Groq to my app on the server: use the groq package (or the OpenAI client with base_url https://api.groq.com/openai/v1) with GROQ_API_KEY from .env, model openai/gpt-oss-20b, retry 429 and 5xx with backoff, and never put the key in browser code.”
Create an API key in the Groq console and keep it in your environment as GROQ_API_KEY.
export GROQ_API_KEY=<your-api-key-here>Install the official Python package (or keep using the OpenAI package with Groq's address).
pip install groqTo reuse OpenAI code, change only the base address and the key.
client = openai.OpenAI( base_url="https://api.groq.com/openai/v1", api_key=os.environ.get("GROQ_API_KEY") )
First request
Trimmed; the full example reply in the docs also lists the settings used. The docs' example reply names llama-3.3-70b-versatile as the model although the request asks for openai/gpt-oss-120b.
curl https://api.groq.com/openai/v1/responses -s \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $GROQ_API_KEY" \
-d '{
"model": "openai/gpt-oss-120b",
"input": "Tell me a three sentence bedtime story about a unicorn."
}'{
"id": "resp_01k1x6w9ane6d8rfxm05cb45yk",
"object": "response",
"status": "completed",
"output": [
{
"type": "message",
"role": "assistant",
"content": [
{
"type": "output_text",
"text": "When the stars blinked awake, Luna the unicorn curled her mane and whispered wishes to the sleeping pine trees. ..."
}
]
}
],
"usage": { "input_tokens": 82, "output_tokens": 266, "total_tokens": 348 }
}Mistakes to avoid
The models page now lists Llama 3.3 70B as Enterprise with Contact Sales pricing.
Anyone who opens the page could copy the key and spend your money.
These OpenAI fields are not supported on Groq.
Browser search does not work together with structured outputs.
The speech-to-text table marks translation as not supported for turbo.
Questions
Is there a free tier?
Yes. The Free tier needs no payment method; you add one only to move to the pay-as-you-go Developer tier.
Can I keep my OpenAI code?
Mostly. Point the OpenAI client at https://api.groq.com/openai/v1 with a Groq key; a few options like logprobs are not supported.
Does Groq train on my prompts?
No. Its agreement says inputs and outputs are not used for training unless you allow it.
Can I cap my bill?
Yes, on paid plans. A spend limit blocks all keys of the organization when the monthly cap is reached.
Am I charged for failed requests?
Not for errors on Groq's side (500, 502, 503).