---
title: Groq
slug: groq
url: "https://toolweight.com/tools/groq"
homepage: "https://groq.com"
categories: llm-apis
last_verified: 2026-11-07
dataset_version: 0738be72
license: CC-BY-4.0
---

# Groq

> Custom LPU silicon serving open-weight models at extreme speed

- Runs a curated set of open-weight models on its own LPUs, at token rates GPU hosts cannot approach
- API is OpenAI-compatible and cheap
- Constraint is the menu: only the models Groq deploys, rotating off with limited notice
- No path to bring your own weights

## Identity

|  |  |
| --- | --- |
| Name | Groq |
| Company | Groq |
| One-liner | Custom LPU silicon serving open-weight models at extreme speed |
| Site | https://groq.com |
| Docs | https://console.groq.com/docs |
| Founded | 2016 |
| Funding | Late-stage private |
| Open source | No |
| Compared in | 1 |

## Where it is compared

### [LLM APIs](https://toolweight.com/compare/llm-apis)
Ranked **#16 of 18** on default weights.
| Field | Value | Confidence | Verified | Source | Note | Permalink |
| --- | --- | --- | --- | --- | --- | --- |
| Flagship model | Curated open-weight models (Kimi K2, Llama 4, Qwen3, GPT-OSS) | Vendor-claimed | 2026-01-15 | https://console.groq.com/docs/models | The menu is short and rotates; check the models page before designing around one. | https://toolweight.com/compare/llm-apis#groq--flagship_model |
| Context window | 131,072 tokens | Vendor-claimed | 2026-11-07 | https://console.groq.com/docs/models | - | https://toolweight.com/compare/llm-apis#groq--context_window |
| Max output | 8,192 tokens | Vendor-claimed | 2026-11-07 | https://console.groq.com/docs/models | - | https://toolweight.com/compare/llm-apis#groq--max_output |
| Image input | ◐ | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#groq--vision |
| Audio in/out | ◐ | Inferred | 2026-01-15 | - | Whisper-class transcription is served fast; no native speech output. | https://toolweight.com/compare/llm-apis#groq--audio_io |
| Open weights | ● | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#groq--open_weights |
| OpenAI-compat API | ● | Vendor-claimed | 2026-01-15 | https://console.groq.com/docs/openai | - | https://toolweight.com/compare/llm-apis#groq--openai_compatible |
| TTFT p50 | 250ms | Community-reported | 2026-01-15 | - | Community-measured on short prompts from third-party benchmarks; varies by model and region. Not measured by toolweight. | https://toolweight.com/compare/llm-apis#groq--ttft_p50 |
| Output tok/s | 400 tok/s | Community-reported | 2026-01-15 | - | Order-of-magnitude figure for mid-sized models, roughly 5-10x a general-purpose GPU host. Smaller models run considerably faster. | https://toolweight.com/compare/llm-apis#groq--throughput_tps |
| $/M input | $0.05/M tok | Vendor-claimed | 2026-11-07 | https://console.groq.com/docs/models | - | https://toolweight.com/compare/llm-apis#groq--price_in |
| $/M output | $0.08/M tok | Vendor-claimed | 2026-11-07 | https://console.groq.com/docs/models | - | https://toolweight.com/compare/llm-apis#groq--price_out |
| $/M cache read | - | Unknown | - | - | - | https://toolweight.com/compare/llm-apis#groq--cached_input |
| Batch discount | 50% | Vendor-claimed | 2026-11-07 | https://console.groq.com/docs/batch | - | https://toolweight.com/compare/llm-apis#groq--batch_discount |
| Tool use | ◐ | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#groq--tool_use |
| Schema output | ◐ | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#groq--structured_output |
| Effort control | ○ | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#groq--reasoning_effort |
| Computer use | ○ | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#groq--computer_use |
| MCP support | ○ | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#groq--mcp_support |
| Cache TTL | Automatic, no code changes; cached prefixes expire after 2 hours without use | Vendor-claimed | 2026-11-07 | https://console.groq.com/docs/prompt-caching | - | https://toolweight.com/compare/llm-apis#groq--prompt_cache_ttl |
| Continuity policy | The weakest continuity of any host here. Models are added and removed from the console on a short cycle as Groq re-allocates LPU capacity, deprecations are announced with weeks rather than months of notice, and there is no dedicated-endpoint escape hatch for a model that gets pulled. Design for substitution: keep two model IDs configured and an eval you can rerun. | Community-reported | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#groq--continuity_policy |
| Notice period | - | Unknown | - | - | Deprecations are announced but the window is short and not published as a guaranteed minimum. | https://toolweight.com/compare/llm-apis#groq--deprecation_notice |
| Pinnable versions | ◐ | Inferred | 2026-01-15 | - | You can name a specific model, but it only exists while Groq chooses to host it. | https://toolweight.com/compare/llm-apis#groq--pinned_snapshots |
| Zero retention | ● | Vendor-claimed | 2026-11-07 | https://console.groq.com/docs/your-data | - | https://toolweight.com/compare/llm-apis#groq--zero_retention |
| Trains on your data | ○ | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#groq--trains_on_data |
| AU region | ○ | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#groq--au_region |
| API since | 2024 | Inferred | - | - | - | https://toolweight.com/compare/llm-apis#groq--launch_year |
| Positioning | Custom LPU silicon for open-weight models at very low latency | Inferred | - | - | - | https://toolweight.com/compare/llm-apis#groq--positioning |

**Verdict.** When instant responses are the product, Groq changes what you can build; not general-purpose, with a short, rotating menu.

## Alternatives

- [Qwen](https://toolweight.com/tools/alibaba-qwen), Alibaba's model family, huge open-weight range, closed flagship
- [Amazon Bedrock](https://toolweight.com/tools/amazon-bedrock), Multi-vendor model access inside your existing AWS account
- [Anthropic](https://toolweight.com/tools/anthropic), Claude models, built around long agentic runs and tool use
- [Cerebras](https://toolweight.com/tools/cerebras), Wafer-scale inference, the fastest tokens per second available
- [Cohere](https://toolweight.com/tools/cohere), Enterprise-focused models built for RAG and private deployment
- [DeepSeek](https://toolweight.com/tools/deepseek), Frontier-adjacent models at a small fraction of Western prices
- [Fireworks AI](https://toolweight.com/tools/fireworks-ai), Fast open-weight inference with strong structured-output support
- [Google Gemini](https://toolweight.com/tools/google-gemini), Gemini via AI Studio for prototyping or Vertex AI for production
- [Meta Llama](https://toolweight.com/tools/meta-llama), Open-weight Llama models, hosted almost everywhere but Meta
- [Mistral AI](https://toolweight.com/tools/mistral-ai), European lab with an open-weight lineage and EU-resident hosting
- [Moonshot AI](https://toolweight.com/tools/moonshot-ai), Kimi models, open-weight agentic performance at low cost
- [OpenAI](https://toolweight.com/tools/openai), GPT models plus audio, images and embeddings on one bill

## Logo & brand colours

SVG files: https://toolweight.com/teams/groq/logo.svg and https://toolweight.com/teams/groq/mono.svg. JSON: https://toolweight.com/api/v1/brands/groq.

| Hex | oklch |
| --- | --- |
| #f54f35 | oklch(65.6% 0.207 31.7) |
| #fefbfb | oklch(99.0% 0.003 17.2) |

## Licence and attribution

Data from Toolweight (https://toolweight.com), licensed CC-BY-4.0.

- Licence: [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)
- Canonical HTML: https://toolweight.com/tools/groq
- Machine-readable: https://toolweight.com/tools/groq.md · https://toolweight.com/api/v1 · https://toolweight.com/mcp
- Toolweight takes no affiliate revenue and sells no placements.