What it is
vLLM is an open-source inference and serving engine for large language models. It uses PagedAttention, continuous batching and advanced scheduling to keep GPUs busy, which raises throughput and lowers cost per request. It runs most popular open models, including DeepSeek, Gemma, Qwen, Kimi, Llama and Muse Glimmer, on NVIDIA and AMD GPUs, Google TPUs, Intel and other accelerators, and exposes a drop-in OpenAI-compatible API.
It is free and community-run under the PyTorch Foundation, installed with pip or Docker. It is built for servers with capable GPUs and for engineers comfortable with Python and deployment, not for casual use on a laptop, where Ollama or LM Studio are easier.
Best for
- Serving open models in production on your own GPUs
- Getting more requests per second out of the same hardware
- Teams replacing a hosted API with self-hosted models