---
title: Self-hosted (vLLM)
slug: self-hosted-vllm
url: "https://toolweight.com/tools/self-hosted-vllm"
homepage: "https://docs.vllm.ai"
categories: llm-apis
last_verified: 2026-11-07
dataset_version: 0738be72
license: CC-BY-4.0
---

# Self-hosted (vLLM)

> Run open weights on your own GPUs behind an OpenAI-shaped API

- The baseline every hosted option is measured against
- vLLM serves open-weight models on your own hardware: OpenAI-compatible endpoint, continuous batching, paged attention, prefix caching
- You trade per-token billing for GPU-hour billing and an operational burden
- The only genuinely permanent answer to deprecation: the weights are on your disk, nobody can retire them

## Identity

|  |  |
| --- | --- |
| Name | Self-hosted (vLLM) |
| Company | Self-hosted (vLLM) |
| One-liner | Run open weights on your own GPUs behind an OpenAI-shaped API |
| Site | https://docs.vllm.ai |
| Repo | https://github.com/vllm-project/vllm |
| Founded | 2023 |
| Open source | Yes |
| Licence | Apache-2.0 |
| Compared in | 1 |

## Where it is compared

### [LLM APIs](https://toolweight.com/compare/llm-apis)
Ranked **#10 of 18** on default weights.
| Field | Value | Confidence | Verified | Source | Note | Permalink |
| --- | --- | --- | --- | --- | --- | --- |
| Flagship model | Whatever you deploy, DeepSeek-V3.x, Qwen3, Kimi K2, Llama 4 | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--flagship_model |
| Context window | - | Unknown | - | - | Set by the checkpoint and by how much KV cache your VRAM budget allows. | https://toolweight.com/compare/llm-apis#self-hosted-vllm--context_window |
| Max output | - | Unknown | - | - | Whatever you configure the server for, bounded by the same VRAM budget as the input window. | https://toolweight.com/compare/llm-apis#self-hosted-vllm--max_output |
| Image input | ◐ | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--vision |
| Audio in/out | ○ | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--audio_io |
| Open weights | ● | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--open_weights |
| OpenAI-compat API | ● | Vendor-claimed | 2026-01-15 | https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html | vLLM serves /v1/chat/completions natively, so client code written against OpenAI works unchanged. | https://toolweight.com/compare/llm-apis#self-hosted-vllm--openai_compatible |
| TTFT p50 | - | Unknown | - | - | Entirely determined by your hardware, batch size and prefix-cache hit rate. | https://toolweight.com/compare/llm-apis#self-hosted-vllm--ttft_p50 |
| Output tok/s | - | Unknown | - | - | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--throughput_tps |
| $/M input | N/A | Vendor-claimed | 2026-11-07 | https://raw.githubusercontent.com/vllm-project/vllm/main/README.md | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--price_in |
| $/M output | N/A | Vendor-claimed | 2026-11-07 | https://raw.githubusercontent.com/vllm-project/vllm/main/README.md | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--price_out |
| $/M cache read | N/A | Vendor-claimed | 2026-11-07 | https://raw.githubusercontent.com/vllm-project/vllm/main/README.md | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--cached_input |
| Batch discount | 0% | Inferred | - | - | Not applicable, continuous batching is how the server works, not a pricing tier. | https://toolweight.com/compare/llm-apis#self-hosted-vllm--batch_discount |
| Tool use | ◐ | Inferred | 2026-01-15 | - | Tool-call parsing depends on a per-model parser plugin and is noticeably more fragile than a hosted API. | https://toolweight.com/compare/llm-apis#self-hosted-vllm--tool_use |
| Schema output | ● | Vendor-claimed | 2026-01-15 | https://docs.vllm.ai/en/latest/features/structured_outputs.html | Guided decoding with JSON Schema, regex or grammar backends. | https://toolweight.com/compare/llm-apis#self-hosted-vllm--structured_output |
| Effort control | ○ | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--reasoning_effort |
| Computer use | ○ | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--computer_use |
| MCP support | ○ | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--mcp_support |
| Cache TTL | Automatic prefix cache in VRAM, no TTL | Vendor-claimed | 2026-01-15 | https://docs.vllm.ai/en/latest/features/automatic_prefix_caching.html | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--prompt_cache_ttl |
| Continuity policy | Permanent, and this is the whole point of the baseline. The checkpoint sits on your disk. Nobody can deprecate it, re-price it, silently upgrade it, or change its behaviour between releases. The obligation transfers to you: you own the GPUs, the upgrades, the security patches and the on-call roster. | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--continuity_policy |
| Notice period | N/A | Vendor-claimed | 2026-11-07 | https://raw.githubusercontent.com/vllm-project/vllm/main/README.md | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--deprecation_notice |
| Pinnable versions | ● | Inferred | 2026-01-15 | - | Pin the checkpoint hash and it is byte-identical forever. | https://toolweight.com/compare/llm-apis#self-hosted-vllm--pinned_snapshots |
| Zero retention | ● | Inferred | 2026-01-15 | - | The one unconditional yes in this column: there is no vendor to retain anything, and no abuse-monitoring log you did not write yourself. | https://toolweight.com/compare/llm-apis#self-hosted-vllm--zero_retention |
| Trains on your data | ○ | Inferred | 2026-01-15 | - | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--trains_on_data |
| AU region | ● | Inferred | 2026-01-15 | - | The only unambiguous answer to an AU-residency requirement. | https://toolweight.com/compare/llm-apis#self-hosted-vllm--au_region |
| API since | 2023 | Inferred | - | - | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--launch_year |
| Positioning | Your weights, your GPUs, your OpenAI-compatible endpoint | Inferred | - | - | - | https://toolweight.com/compare/llm-apis#self-hosted-vllm--positioning |

**Verdict.** A reference point, not a recommendation; it wins on cost only at high steady load, so choose it for sovereignty or permanence.

## Alternatives

- [Qwen](https://toolweight.com/tools/alibaba-qwen), Alibaba's model family, huge open-weight range, closed flagship
- [Amazon Bedrock](https://toolweight.com/tools/amazon-bedrock), Multi-vendor model access inside your existing AWS account
- [Anthropic](https://toolweight.com/tools/anthropic), Claude models, built around long agentic runs and tool use
- [Cerebras](https://toolweight.com/tools/cerebras), Wafer-scale inference, the fastest tokens per second available
- [Cohere](https://toolweight.com/tools/cohere), Enterprise-focused models built for RAG and private deployment
- [DeepSeek](https://toolweight.com/tools/deepseek), Frontier-adjacent models at a small fraction of Western prices
- [Fireworks AI](https://toolweight.com/tools/fireworks-ai), Fast open-weight inference with strong structured-output support
- [Google Gemini](https://toolweight.com/tools/google-gemini), Gemini via AI Studio for prototyping or Vertex AI for production
- [Groq](https://toolweight.com/tools/groq), Custom LPU silicon serving open-weight models at extreme speed
- [Meta Llama](https://toolweight.com/tools/meta-llama), Open-weight Llama models, hosted almost everywhere but Meta
- [Mistral AI](https://toolweight.com/tools/mistral-ai), European lab with an open-weight lineage and EU-resident hosting
- [Moonshot AI](https://toolweight.com/tools/moonshot-ai), Kimi models, open-weight agentic performance at low cost

## Licence and attribution

Data from Toolweight (https://toolweight.com), licensed CC-BY-4.0.

- Licence: [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)
- Canonical HTML: https://toolweight.com/tools/self-hosted-vllm
- Machine-readable: https://toolweight.com/tools/self-hosted-vllm.md · https://toolweight.com/api/v1 · https://toolweight.com/mcp
- Toolweight takes no affiliate revenue and sells no placements.