> ## Documentation Index
> Fetch the complete documentation index at: https://help.nops.io/llms.txt
> Use this file to discover all available pages before exploring further.

# LiteLLM

> Connect a self-hosted LiteLLM gateway to nOps Inform

## Overview

nOps connects to your **LiteLLM proxy** with a **read API key** to see models, keys, and spend across the gateway. You can also add an optional **write API key** for future management actions (keys, teams, budgets). Keys are validated before save and are never shown again after you connect. nOps shows only the last four characters of each key.

You can connect more than one gateway. Each gateway is a separate connection with its own base URL and keys.

## Prerequisites

* A LiteLLM proxy you administer, with the admin UI or the management API enabled
* The gateway reachable from the internet over **public HTTPS** (recommended) or HTTP (see [Network requirements](#network-requirements))
* nOps organization **administrator** access

> [!NOTE]
>
> Only **admins** and **owners** can connect, re-validate, or disconnect. **Members** see **Admin Only** on connectable cards.

## Create the read API key

The read key needs to read model info, keys, and spend for the whole gateway. Use a key that belongs to a **`proxy_admin_viewer`** user, so it sees every team but cannot change anything, and limit it to the read routes below.

1. Create a viewer user (skip this if you already have one):

   ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
   curl -X POST https://litellm.example.com/user/new \
     -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
     -H "Content-Type: application/json" \
     -d '{"user_email": "nops@example.com", "user_role": "proxy_admin_viewer"}'
   ```

2. Create a key for that user that can call only these read routes:

   ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
   curl -X POST https://litellm.example.com/key/generate \
     -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "user_id": "<user_id from step 1>",
       "key_alias": "nops-read",
       "allowed_routes": [
         "/model/info",
         "/key/info",
         "/key/list",
         "/user/daily/activity",
         "/spend/logs",
         "/spend/tags",
         "/global/spend/report",
         "/team/list",
         "/team/info",
         "/user/info"
       ]
     }'
   ```

3. Copy the `key` value from the response. LiteLLM shows it only once.

> [!WARNING]
>
> Do not use `"key_type": "read_only"` (or `"allowed_routes": ["info_routes"]`). That preset blocks `/user/daily/activity`, which nOps needs to read spend, so spend shows as **Denied**.

> [!WARNING]
>
> Do not use an **LLM API** key (`key_type: llm_api`) or a key scoped to one team. An LLM API key can only call model routes, and a team key shows nOps only that team's usage. nOps warns you after it validates the key.

## Create the write API key (optional)

Add a write key only if you want nOps to manage LiteLLM keys, teams, or budgets later. Create it for a **`proxy_admin`** user, or use `"key_type": "management"`. Keep it separate from the read key.

When you save a write key, nOps checks which management routes it can call with requests that change nothing: each one targets an id that does not exist (`nops-probe-…`), so LiteLLM answers "not found" instead of making a change. If the key cannot call any management route, nOps does not save it.

## Connect in nOps

1. Go to **Settings → Integrations** and click the **LiteLLM** card.
2. Enter the **Base URL** of the proxy (for example `https://litellm.example.com`). Use the final URL; nOps does not follow redirects. If the URL starts with `http://`, confirm **I understand API keys will be sent unencrypted** to continue.
3. Paste the **Read API key** and, optionally, the **Write API key**.
4. Optionally add a **label** (for example `Production gateway`).
5. Click **Connect**.

nOps checks that the gateway is healthy, that each key is accepted, and which routes each key can call. Nothing is saved if a check fails.

## Savings opportunities

Open a gateway in **Settings → Integrations → LiteLLM** and click **Savings**. nOps reads the last 14 days of spend through your read key and looks for prompt-caching opportunities: models, such as Claude, that only cache when requests carry cache markers and where little of your prompt traffic is served from the provider's cache.

Each recommendation shows:

* A **monthly savings range**, not a single number. It is based on your observed cache reads and the model's own cache prices, and assumes only a modest share of the remaining prompt tokens become cache reads.
* The **evidence** behind it: prompt tokens, cache-read share, and spend over the period.
* **How to apply**: the `cache_control_injection_points` setting to add to that model.

> [!NOTE]
>
> nOps does not change your gateway from this page. Applying the setting is up to you, and it affects every team that uses that model. If your gateway stores models in its database, changes take about a minute to go live.

Models that already cache most of their prompts, or that nOps cannot price, are left out. If nothing qualifies, nOps says so and shows your overall cache-read share.

## Network requirements

* The base URL should use **`https://`** with a public hostname. IP addresses, `localhost`, and `.local` / `.internal` hostnames are rejected.
* **`http://`** is allowed only after an admin explicitly confirms it, because API keys and responses then travel unencrypted. nOps shows an **Insecure (http)** badge and a warning on the connection. Switch to https when possible. Moving a connection from https to http needs the same confirmation and the keys re-entered.
* http does not change the network rules below: private networks are still unreachable over http.
* The hostname must resolve only to **public** IP addresses. nOps checks DNS before every request and refuses private, loopback, link-local, and carrier-grade NAT ranges.
* Redirects are not followed. If the proxy redirects (for example from `/` to `/ui`, or from `http://` to `https://`), enter the URL that answers directly.
* If the gateway only accepts traffic from allowlisted IP addresses, contact nOps support before connecting.

## Manage your connection

* **Re-validate** re-runs the checks with the stored keys. A failure shows **Action required**.
* **Rotate a key** by entering a new value; leave a key field unchanged to keep the stored key.
* **Remove the write key** to go back to read-only access.
* **Change the base URL** on a different host only by re-entering the read key (and the write key, or removing it). nOps never sends stored keys to a new host.
* **Disconnect** deletes the stored keys.

## Troubleshooting

* **Could not reach the gateway:** confirm the base URL is publicly reachable and that `/health/liveliness` returns 200.
* **Host is not allowed:** the hostname resolves to a private or reserved address. Expose the proxy on a public address.
* **LiteLLM redirected:** enter the final URL the proxy serves, without redirects. A redirect from http to https means you should enter the `https://` URL.
* **Confirm insecure http:** the URL starts with `http://`. Use the `https://` URL, or confirm that keys will be sent unencrypted.
* **Read key rejected:** confirm the key is active and was copied in full.
* **Read key cannot read spend or keys:** the key is missing routes, most often `/user/daily/activity` (spend) or `/key/list` (keys). Add them to the key's `allowed_routes` (`POST /key/update`) or recreate it as described in [Create the read API key](#create-the-read-api-key), for a `proxy_admin_viewer` user.
* **Write key cannot call any management routes:** the write key's user is not an admin, or the key's routes exclude management routes.

## Security

Treat both keys like passwords. Rotate them in LiteLLM if they may have been exposed, then update them in nOps.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.