LLM Deployment Strategies: APIs, Self-Hosting, and Everything Between

Serving an LLM is a menu with five entries, and the right answer is rarely "the biggest GPU you can rent." This guide maps the options — managed APIs, serverless, dedicated GPUs, self-hosted fleets, and hybrids — with the trade-offs that actually decide outcomes: cost, latency, data control, and operational burden.

The menu

Managed APIs: the default starting point

The case for APIs is not just convenience — it's economics. At low to medium volume, the fully-loaded cost of self-hosting (GPUs, failover, engineers, evaluation) exceeds per-token API pricing. APIs also deliver instant scale, automatic versioning, and a security boundary you don't maintain.

The costs to accept: data leaves your boundary (relevant for regulated data — see the AI security guide), per-token spend grows linearly with usage, and you depend on a vendor's uptime and roadmap.

Self-hosting: when it earns its keep

Self-hosting wins in four situations:

1. Sustained high volume — above roughly 1M tokens/day, fixed GPU cost amortizes below per-token pricing.
2. Data residency — the data cannot leave the boundary, full stop.
3. Latency at the edge — in-region self-hosted inference beats cross-region APIs.
4. Fine-tuned models — custom weights that the API vendors don't serve.

The honest costs: GPU fleet operations (orchestration guide), model versioning and rollback, and the evaluation discipline to know when a "better" model is actually worse.

The hybrid: routing as the answer

Most production systems end up hybrid — the model routing pattern:

request → router (classifier)
    ├─ simple task      → small open model (self-hosted or serverless)
    ├─ standard task    → managed API (medium model)
    └─ hard reasoning   → largest model (API or fleet)

Routing blends cost and control: sensitive data routes to the boundary, commodity traffic routes to the cheapest adequate model, and the whole thing is invisible to callers.

Choosing models: evals decide

Whatever the deployment path, model choice is an evaluation problem:

1. Build a task-specific eval set (50–200 representative inputs with expected outputs).
2. Run candidate models through it — including the small ones; they win more often than assumed.
3. Measure quality, latency, and cost per request; choose on the blend.
4. Re-run on every model version change — the observability guide covers the CI integration.

The migration path

The sequence that avoids regret:

1. API + evals + instrumentation      (week 1–2)
2. Add routing; cache aggressively    (month 1)
3. Pilot self-host for ONE workload   (month 2–3)
4. Expand the fleet only on evidence  (after the pilot proves cost/latency)

Cost modeling: a worked example

The build-versus-rent decision is a spreadsheet, not a slogan. Here is the shape of it. Suppose a workload sends 2M tokens/day, or ~60M tokens/month, at an average price of $4 per million tokens (roughly the mid-range of a good API model):

• API cost: 60M × $4/M = ~$240/month. Zero ops, instant scale, your only cost is usage.
• Self-hosted cost: one A100-class GPU at ~$1.50/hour = ~$1,080/month rental, plus engineers to operate it, plus failover, plus the risk of idle time. That exceeds the API bill until utilization is high and sustained.
• The crossover: as volume grows, the fixed GPU cost amortizes and per-token cost drops. The crossover for most workloads lands somewhere in the millions of tokens per day — but only if the fleet actually runs at 60–80% utilization. Under-utilized, self-hosting never crosses over.

The discipline: re-run this spreadsheet quarterly as volume, model prices, and hardware costs change. A deployment that made sense in January can flip by June.

Provider lock-in and portability

Any real deployment needs an exit path from any single vendor:

• Abstract the interface. Call models through a thin, provider-neutral layer (OpenAI-compatible endpoints are the de facto standard), so swapping providers is a config change, not a rewrite.
• Pin what you must, abstract what you can. Pin model versions for quality and reproducibility, but keep the calls portable. The tension is real and intentional — portability at the interface, determinism at the model layer.
• Keep evals provider-agnostic. Your eval set and scoring should run against any candidate model, so a switch is decided on the same evidence, not on marketing (observability guide).
• Test the switch early. Do a real pilot through a second provider on a low-stakes workload before you need it under pressure. A portability claim that has never been exercised is a hope, not a capability.

The deployment runbook

Whatever the target, every deployment should pass the same checks: a task-specific eval gate in CI, pinned model version, redacted logging and tracing enabled, rate limits and quotas set, secrets out of prompts, and a documented rollback that restores the previous model version and prompt in one action. If rollback is not a single step, it is not a rollback.

Key takeaways

• Deployment is a menu, not a religion: managed APIs for speed, self-hosting for volume and control, hybrids for the real world.
• The build-versus-rent decision is a spreadsheet re-run quarterly — the crossover sits at sustained, high-utilization volume.
• Keep the interface portable and the model version pinned; test a second-provider pilot before you need it under pressure.
• Evals, routing, and instrumentation are the discipline that makes any deployment path work.

Ownership and decision records

Whatever option you choose, record the decision and the evidence: the workload volume, the crossover math, the eval scores, and who owns the deployment. Six months later, when someone asks why a workload is self-hosted on GPUs instead of an API, the decision record turns a debate into a review.

Conclusion

Conclusion

LLM deployment is a menu, not a religion: APIs for speed and economics, self-hosting for volume and control, hybrids for the real world. The discipline that makes any choice work is the same — evals, routing, and instrumentation. The inference optimization guide makes the chosen path fast and cheap; the security guide keeps it safe.