Skip to main content

Databricks BC+ Integration

Integrate your Databricks billing data with the nOps platform to gain detailed insights into your cloud expenses. By setting up this integration, you can analyze your Databricks cost and usage information using the Cost Analysis tool in the nOps platform.

How It Works​

  1. Integration Setup

    • Identify the S3 bucket in your AWS account.
    • Deploy the CloudFormation stack provided by nOps to grant GetObject and ListBucket permissions to the bucket and the required prefix.
    note

    Make sure to be logged into the correct AWS account to proceed with this step.

  2. Data Upload from Databricks

    • nOps provides a Python script that extracts billing data from your Databricks workspace.
    • Schedule this script as a job in your Databricks environment to periodically upload the billing data to the configured S3 bucket.

  3. Data Retrieval by nOps

    • nOps fetches the billing files from the specified bucket and prefix using the granted permissions.

  4. Data Processing

    • The retrieved data is processed and displayed within the Business Context+ product, providing actionable insights into your Databricks usage.

Benefits of Integration​

  • Centralized Cost Visibility
    Access a unified view of your Databricks expenses alongside other cloud cost data in the nOps platform.

  • Resource Optimization
    Identify high-cost areas and optimize resource allocation for better efficiency.

  • Enhanced Transparency
    Gain a clear understanding of your Databricks usage trends and cost breakdowns.

By leveraging the BC+ Databricks integration with nOps, you can effectively monitor and control your Databricks expenses, ensuring cost transparency and improved resource management.