What is OSW

OSW is an AI gateway that runs on your own machine. It starts a local HTTP service, and all your AI clients point at that one address; requests are forwarded through it to the upstream model services you configure, and when an upstream fails, the next one is tried automatically.

It solves a very concrete problem: when you have several model services (different vendors, different keys, different quotas), each client carries its own pile of addresses and secrets, and when one goes down you fix them by hand. OSW gathers all of that into one layer, so clients only ever need to know a single address.

One address, many upstreams

Clients always point at http://127.0.0.1:9300. Swap upstreams or add more, and clients never change.

Automatic failover

Network errors, timeouts, rate limits, and 5xx responses fail over to the next candidate; the successful attempt still returns its result to the client.

Per-request routing

Use rules or a workflow to decide which logical model a request lands on: split by source, model name, or protocol.

Visible by default

Which upstream each request took, how many times it failed over, and where the time went are all recorded in request logs and analytics.

The journey of a request

flowchart LR
  A[Client] -->|one address| B[OSW local service]
  B --> C{Routing}
  C -->|logical model| D[Candidate upstream 1]
  D -.failed.-> E[Candidate upstream 2]
  D -->|success| F[Back to client]
  E -->|success| F

The client sends a request to the local address (the protocol can be OpenAI or Anthropic — see Supported protocols).
OSW uses the currently active routing definition to decide which logical model the request lands on.
Under that logical model hang several provider models (real upstreams); one is chosen by priority / failover to make the request.
When an upstream fails in a retryable way, the next candidate is tried; a successful response (including streaming) is handed back to the client as-is.

What it is not

OSW is a local gateway. It does not run as a cloud service, and it does not take over your system network. A few boundaries up front:

Boundaries

No system-level proxy: clients must explicitly point at the local service; OSW does not hijack system traffic.
No client-side auth check: any non-empty key works at the local address. The real secret is substituted by OSW when forwarding.
Cross-upstream protocol conversion is a compatibility layer: only endpoints where you explicitly enable conversion are eligible for cross-protocol requests, and some parameters may be lost (see Supported protocols).

Ready to go further? Start with Install & run, or jump straight to Three steps to a gateway.