← AUTOMATER NEWSROOM

Vercel AI Gateway Adds Private Inference: Your Fine-Tuned and Self-Hosted Models Behind the Same Key

Vercel AI Gateway Private Inference registers your own model endpoints as private/<slug> models behind the same keys, coding agents and observability as the catalog.

Diagram of Vercel AI Gateway Private Inference: a private model endpoint and a public catalog both routing through one gateway box to apps and coding agents.Diagram of Vercel AI Gateway Private Inference: a private model endpoint and a public catalog both routing through one gateway box to apps and coding agents.
Original illustration: a private model endpoint and the public catalog both route through Vercel AI Gateway, which hands apps and coding agents one key and unified logs. Concept based on the Vercel changelog.

Vercel has turned AI Gateway from a router for other people’s models into a control plane for all of your models. Private Inference, announced on the Vercel changelog on Nov. 16, lets a team register its own model endpoints — fine-tuned checkpoints, self-hosted open-weight servers, prerelease builds — and call them exactly like the 400+ models in the public catalog: same gateway key, same AI SDK or Chat Completions/Responses/Messages call, same logs and spend charts.

Vercel changelog page announcing that AI Gateway now supports Private Inference, showing the announcement text and setup details.
Vercel's changelog entry for AI Gateway Private Inference, published Nov. 16, 2026, describing how private model endpoints use the same authentication, SDKs and observability as catalog models. · Original source

What actually changed

Before this, a team running a vLLM box next to frontier APIs maintained two stacks: gateway keys and dashboards for catalog models, plus separate credentials, client code and log scraping for the private endpoint. The changelog is explicit about the motivation: enterprises running fine-tuned, self-hosted or prerelease models “used to need separate keys, client code, and logs for them.”

Now you register the endpoint once as a private provider — base URL, API format, credentials — then add the models it serves. Registration works in the dashboard’s Private Inference page or via the Vercel CLI from version 65.16.0:

# Connect your endpoint as a private provider
vercel ai-gateway private-inference providers create acme-vllm \
  --base-url https://vllm.example.com/v1

# Expose the endpoint's model as private/support-fine-tune
vercel ai-gateway private-inference models create support-fine-tune \
  --provider acme-vllm \
  --api-model acme/support-llama-ft

The model then gets a private/<slug> ID. Calling it is a one-line change in existing code:

import { streamText } from 'ai';

const result = streamText({
  model: 'private/support-fine-tune',
  prompt: 'Draft a reply to a customer who was charged twice.',
  providerOptions: {
    gateway: {
      // Optional fallback to a catalog model
      models: ['anthropic/claude-sonnet-5.5'],
    },
  },
});

The gateway translates each request into your endpoint’s format, so switching between a private fine-tune and a catalog model is only a model-ID change. If the private endpoint fails, an optional fallback routes the request to a catalog model instead of returning an error — the same failover behavior the gateway already applies across public providers.

Scoping and data handling

Private models are visible and callable only inside your Vercel team and never appear in the public catalog. Provider credentials are stored encrypted, and Zero Data Retention can be enabled per provider — relevant for teams whose fine-tunes are trained on customer data, since the same security posture the gateway offers for catalog routing (ZDR, no-training routing, provider allowlists) now extends to your own endpoints.

Why it matters for coding-agent fleets

The practical audience is operators running mixed fleets: a frontier model for hard work, an internal fine-tune for support replies or code migration, an open-weight server for cheap bulk passes. AI Gateway already routes 11+ coding agents — Claude Code, Codex, Cursor, OpenCode, Cline and others — with a base URL change and a setup command that provisions keys and writes agent configs. Private Inference means Claude Code can now sit in front of your self-hosted model with the same AI_GATEWAY_API_KEY it uses for Anthropic, and the request shows up in the same logs.

That is the quietly significant part. Observability charts break down requests, tokens and time-to-first-token by private model, and private traffic appears alongside catalog traffic in logs, trace drains and custom reporting. Budgets and rate limits that already cap your catalog spend now bound your fine-tune traffic too. The gateway — which Vercel says is a hosted API usable from any cloud with zero markup on token prices — becomes the single place where model access, spend and audit records live, regardless of who serves the weights. That is exactly the consolidation our agent gateway control plane analysis argued enterprises were heading toward: one layer owning identity, routing and the audit trail for every model an agent touches.

For teams that previously kept a LiteLLM proxy or hand-rolled shim beside the gateway for private endpoints, this collapses two control planes into one. The trade is dependency: your private endpoint’s availability and cost controls now route through Vercel’s service, so the fallback configuration is worth setting on day one.

Capability Before Private Inference After
Calling a self-hosted model Separate base URL, key and client code private/<slug> via existing gateway key
Switching to a catalog model Code changes in the client Change the model ID
Endpoint failure Custom retry logic Optional catalog-model fallback
Logs and spend Separate dashboard or scraper Same logs, charts and budgets
Coding agents Catalog models only Private models in Claude Code, Codex, Cursor

This continues the direction Vercel has been building all fall — virtual models gave tasks stable IDs, session affinity added per-session cost tracking, and Private Inference closes the last gap by letting your own weights join the same routing, billing and observability fabric.

Getting started

Open the Private Inference page in the Vercel dashboard, add your endpoint’s base URL, API format and credentials, then declare the models it serves. Read the Private Inference documentation for supported model types, requirements and billing details before pointing production traffic at it. The CLI path requires Vercel CLI 65.16.0 or later.

Sources

  1. AI Gateway now supports Private Inference - Vercel
  2. AI Gateway - Vercel
  3. Vercel AI Gateway: Models, Routing, and Observability