About Groq
Groq runs open language models on custom hardware built for inference rather than training, which is why tokens arrive much faster than on a normal GPU stack. The API is OpenAI compatible, so switching a project over is mostly a base URL change.
Comments (3)
- Kenji L.3mo ago
small thing, but the speed is genuinely different. you watch the response appear instantly and it changes what you build...
- hqyilmaz3mo ago
Openai compatible endpoint so switching is a base url change. Took me four minutes. Not glamorous, which is a compliment here.
- Fiona Evans2mo ago
The thing that held up for us: Model selection is limited to what they have deployed. That is the tradeoff for the speed.
Sign in to leave a comment.
