What it is
Groq runs an inference cloud built on its LPU chips, now alongside NVIDIA GPUs, and sells access to open models through an OpenAI-compatible API. Production models include OpenAI's gpt-oss 120B and 20B, Llama 3.1 and 3.3, and Whisper for speech-to-text, with tool use, structured outputs, prompt caching, batch processing and built-in tools such as code execution and browser search.
A Free tier lets you start with lower rate limits; the Developer tier adds higher limits, batch and flex processing, billed pay as you go per token. The model list is shorter than broader hosts. Nvidia licensed Groq's chip technology and hired its founding CEO in December 2025, and Groq now operates as an independent inference cloud.
Best for
- Voice agents and chat apps where response speed matters
- Running open models like gpt-oss and Llama through an API
- Fast Whisper speech-to-text