---
title: Self-hosted (vLLM)
slug: self-hosted-vllm
url: "https://toolweight.com/options/self-hosted-vllm"
homepage: "https://docs.vllm.ai"
categories: llm-apis
last_verified: 2026-01-15
license: CC-BY-4.0
---

# Self-hosted (vLLM)

> Run open weights on your own GPUs behind an OpenAI-shaped API

The baseline every hosted option is measured against. vLLM serves open-weight models on your own hardware with an OpenAI-compatible endpoint, continuous batching, paged attention and prefix caching. You trade per-token billing for GPU-hour billing and an operational burden, and you get the only genuinely permanent answer to deprecation: the weights are on your disk and nobody can retire them.

## Identity

|  |  |
| --- | --- |
| Name | Self-hosted (vLLM) |
| Company | Self-hosted (vLLM) |
| One-liner | Run open weights on your own GPUs behind an OpenAI-shaped API |
| Site | https://docs.vllm.ai |
| Repo | https://github.com/vllm-project/vllm |
| Founded | 2023 |
| Open source | Yes |
| Licence | Apache-2.0 |
| Compared in | 1 |

## Where it is compared

### [LLM APIs](https://toolweight.com/compare/llm-apis)
Ranked **#7 of 18** on default weights.
| Field | Value | Confidence | Verified | Source | Note |
| --- | --- | --- | --- | --- | --- |
| Flagship model | Whatever you deploy, DeepSeek-V3.x, Qwen3, Kimi K2, Llama 4 | Inferred | 2026-01-15 | - | - |
| Context window | - | Unknown | - | - | Set by the checkpoint and by how much KV cache your VRAM budget allows. |
| Max output | - | Unknown | - | - | Whatever you configure the server for, bounded by the same VRAM budget as the input window. |
| Image input | ◐ | Inferred | 2026-01-15 | - | - |
| Audio in/out | ○ | Inferred | 2026-01-15 | - | - |
| Open weights | ● | Inferred | 2026-01-15 | - | - |
| OpenAI-compat API | ● | Vendor-claimed | 2026-01-15 | https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html | vLLM serves /v1/chat/completions natively, so client code written against OpenAI works unchanged. |
| TTFT p50 | - | Unknown | - | - | Entirely determined by your hardware, batch size and prefix-cache hit rate. |
| Output tok/s | - | Unknown | - | - | - |
| $/M input | - | Unknown | - | - | You pay GPU-hours, not tokens. An 8xH100 node runs roughly US$20-30/hour on demand, so the per-token cost depends entirely on utilisation, cheap at high load, ruinous at low. |
| $/M output | - | Unknown | - | - | - |
| $/M cache read | - | Unknown | - | - | Prefix caching is free and lives in VRAM; there is no separate cache-read rate. |
| Batch discount | 0 % | Inferred | - | - | Not applicable, continuous batching is how the server works, not a pricing tier. |
| Tool use | ◐ | Inferred | 2026-01-15 | - | Tool-call parsing depends on a per-model parser plugin and is noticeably more fragile than a hosted API. |
| Schema output | ● | Vendor-claimed | 2026-01-15 | https://docs.vllm.ai/en/latest/features/structured_outputs.html | Guided decoding with JSON Schema, regex or grammar backends. |
| Effort control | ○ | Inferred | 2026-01-15 | - | - |
| Computer use | ○ | Inferred | 2026-01-15 | - | - |
| MCP support | ○ | Inferred | 2026-01-15 | - | - |
| Cache TTL | Automatic prefix cache in VRAM, no TTL | Vendor-claimed | 2026-01-15 | https://docs.vllm.ai/en/latest/features/automatic_prefix_caching.html | - |
| Continuity policy | Permanent, and this is the whole point of the baseline. The checkpoint sits on your disk. Nobody can deprecate it, re-price it, silently upgrade it, or change its behaviour between releases. The obligation transfers to you: you own the GPUs, the upgrades, the security patches and the on-call roster. | Inferred | 2026-01-15 | - | - |
| Notice period | - | Unknown | - | - | Not applicable, there is no vendor to give notice. |
| Pinnable versions | ● | Inferred | 2026-01-15 | - | Pin the checkpoint hash and it is byte-identical forever. |
| Zero retention | ● | Inferred | 2026-01-15 | - | The one unconditional yes in this column: there is no vendor to retain anything, and no abuse-monitoring log you did not write yourself. |
| Trains on your data | ○ | Inferred | 2026-01-15 | - | - |
| AU region | ● | Inferred | 2026-01-15 | - | The only unambiguous answer to an AU-residency requirement. |
| API since | 2023 | Inferred | - | - | - |
| Positioning | Your weights, your GPUs, your OpenAI-compatible endpoint | Inferred | - | - | - |

**Verdict.** The reference point rather than a recommendation. Self-hosting only beats a hosted API on cost at genuinely high, steady utilisation, and it costs you the agentic features, effort control, computer use, MCP, that make the frontier APIs worth their price. Choose it for sovereignty or permanence, not to save money.

## Alternatives

- [Qwen](https://toolweight.com/options/alibaba-qwen), Alibaba's model family, huge open-weight range, closed flagship
- [Amazon Bedrock](https://toolweight.com/options/amazon-bedrock), Multi-vendor model access inside your existing AWS account
- [Anthropic](https://toolweight.com/options/anthropic), Claude models, built around long agentic runs and tool use
- [Cerebras](https://toolweight.com/options/cerebras), Wafer-scale inference, the fastest tokens per second available
- [Cohere](https://toolweight.com/options/cohere), Enterprise-focused models built for RAG and private deployment
- [DeepSeek](https://toolweight.com/options/deepseek), Frontier-adjacent models at a small fraction of Western prices
- [Fireworks AI](https://toolweight.com/options/fireworks-ai), Fast open-weight inference with strong structured-output support
- [Google Gemini](https://toolweight.com/options/google-gemini), Gemini via AI Studio for prototyping or Vertex AI for production
- [Groq](https://toolweight.com/options/groq), Custom LPU silicon serving open-weight models at extreme speed
- [Meta Llama](https://toolweight.com/options/meta-llama), Open-weight Llama models, hosted almost everywhere but Meta
- [Mistral AI](https://toolweight.com/options/mistral-ai), European lab with an open-weight lineage and EU-resident hosting
- [Moonshot AI](https://toolweight.com/options/moonshot-ai), Kimi models, open-weight agentic performance at low cost

## Licence and attribution

Data from toolweight (https://toolweight.com), licensed CC-BY-4.0.

- Licence: [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)
- Canonical HTML: https://toolweight.com/options/self-hosted-vllm
- Machine-readable: https://toolweight.com/options/self-hosted-vllm.md · https://toolweight.com/api/v1 · https://toolweight.com/mcp
- toolweight takes no affiliate revenue and sells no placements. Corrections: https://toolweight.com/suggest