> ## Documentation Index
> Fetch the complete documentation index at: https://help.nops.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Databricks

> Export Databricks system-table billing data to S3 for nOps Inform

## Overview

nOps reads **Databricks billing data** from `system.billing.usage` and `system.billing.list_prices` after you export those tables to S3. Databricks is an **export-only** Settings integration: you choose an S3 destination in nOps, deploy a prefix-scoped read-only role, then import and schedule a generated notebook in your Databricks workspace.

nOps does **not** create or run Databricks jobs, and does **not** collect a workspace URL, SQL warehouse ID, or machine credential.

You can connect **one** AWS Databricks export per organization. Azure Databricks export is not available yet.

<Note>
  This flow is separate from AWS cloud onboarding and does **not** create a Cost and Usage Report (CUR).
</Note>

## Prerequisites

* nOps organization **administrator** or **owner** access
* Permission to deploy CloudFormation (including IAM roles) in the AWS account that holds the export bucket
* Administrative access to a Databricks workspace so you can import a notebook, grant S3 write (for example a Unity Catalog external location), and schedule a daily job
* Either permission for CloudFormation to create a new S3 bucket, or an existing bucket in the same AWS account (or separately grant that account access)

<Note>
  Only **admins** and **owners** can connect, view instructions, or disconnect. **Members** see **Admin Only** on connectable cards.
</Note>

## Connect in nOps

### Where to open Databricks

* **To connect for the first time** — go to **Settings → Integrations** and click the **Databricks** card.
* **To manage an existing export** — open the same card or use **Settings → Account Status → Connected apps** and click the **Databricks** chip.

### Connect steps

1. Click the **Databricks** card. The AWS tab is the supported path.
2. Choose **New S3 bucket** (CloudFormation creates the bucket and nOps read role) or **Existing S3 bucket** (CloudFormation creates a read role scoped to your prefix).
3. Enter the **S3 bucket name** and a **dedicated prefix** (default `nops/databricks`). Do not reuse a shared prefix or a CUR prefix. The notebook writes usage and list-prices Parquet under that prefix and replaces only its own year/month/table/client path.
4. **Download .py notebook** or **Copy notebook**. nOps fills in the S3 destination, nOps client ID, and organization name. The generated file exports `system.billing.usage` and `system.billing.list_prices` and is ready to import as a Databricks notebook.
5. Click **Save and open CloudFormation**. Deploy the stack in the customer AWS account. It creates a prefix-scoped, read-only role that the existing `nops-data-connector` can assume. The stack does not modify an existing bucket policy.
6. After the stack finishes, paste the **deployed role ARN** and click **Save deployed role**.

<Warning>
  Use a dedicated prefix. A shared prefix can overwrite other data under that path. The export is not CUR.
</Warning>

## Import and schedule the notebook in Databricks

nOps does not create, run, or manage jobs in your workspace. After you download or copy the notebook in nOps:

1. Log in to your Databricks workspace.
2. Import the generated `.py` file (or create a notebook and paste the copied script). You can name it something like `NopsDatabricksBillingDataUploader`.
3. Grant this workspace permission to **write** to the S3 destination (for example a Unity Catalog external location). The CloudFormation role is for nOps **read** access only.
4. In the notebook toolbar, click **Schedule**.
5. Select a compute cluster and set the frequency to **Every 1 day**.
6. Create the schedule and confirm a run completes.

Daily scheduling keeps Inform current. The first sync can take up to **24 hours**.

## Azure Databricks

Coming soon. Azure Databricks onboarding will use an Azure-native export storage and identity setup. No AWS CloudFormation step is available for Azure.

## Troubleshooting

* **Shared or CUR prefix** — use a dedicated prefix such as `nops/databricks`. The notebook overwrites only its own year/month/table/client path under that prefix.
* **No data after about 24 hours** — confirm the scheduled notebook job is succeeding, objects exist under the prefix, and the deployed role ARN is saved in nOps.
* **Notebook cannot write to S3** — grant Unity Catalog (or equivalent) write access to the export bucket and prefix. The nOps CloudFormation role does not grant Databricks write.
* **CloudFormation / role errors** — deploy in the AWS account that owns the bucket; paste the role ARN after the stack completes. Existing buckets must be in that account (or separately grant access). The stack does not update an existing bucket policy.
* **Azure tab** — Azure export is not available yet. Use AWS.

## Security

nOps stores only the non-secret AWS destination and cross-account role configuration. The CloudFormation stack creates a prefix-scoped, read-only role trusted by `nops-data-connector` with your external ID. nOps never stores your Databricks workspace URL, warehouse ID, machine credential, or job configuration. The customer-run notebook writes billing Parquet to your bucket; nOps reads only that prefix.