Skip to main content
Databricks on AWS runs its compute on EC2 instances in your account. Graviton instance types cost less per hour than the same-size x86 types, and Databricks supports them on recent runtimes. nOps finds the EC2 instances Databricks launched on x86 types that have a Graviton equivalent and estimates the EC2 saving from switching.

Why it matters

Databricks node types are usually picked once in a cluster or job template and reused everywhere, so one change can lower the cost of many runs.

How nOps finds it

nOps reads the last 7 days of on-demand EC2 billing for instances tagged as launched by Databricks, grouped by job, SQL warehouse, instance pool or cluster and by instance type. Instances in the m5 to m7, r5 to r7 and c5 to c7 x86 families (including the ā€œdā€ variants with local disk) are compared with the same size in the 6g and 7g Graviton families, and i3 with i4g. The saving is the on-demand EC2 cost times the list-price difference to the cheaper Graviton type in the same region. Databricks DBU charges are not included. Hours covered by Reserved Instances or Savings Plans and savings under $5 a month are left out.

Before you start

A Databricks Runtime version that supports Graviton, and libraries or init scripts that work on Arm64.

How to implement

  1. Check the cluster, job or pool runs a Databricks Runtime version that supports Graviton, and that its libraries and init scripts work on Arm64.
  2. Edit the compute and pick the recommended Graviton node type (for example m6gd or r7gd) for workers and the driver.
  3. For jobs, change the node type in the job cluster definition; for instance pools, create a pool with the Graviton type and point clusters at it.
  4. Run the job or a test query and compare run time.
  5. Update cluster policies, Terraform or Databricks Asset Bundles so new clusters use the Graviton type.

Risks and considerations

  • Some features and libraries are not available on Graviton nodes. Test before switching production jobs.
  • Run time can change, which changes DBU cost in either direction.
  • To roll back, set the node type back to the x86 type.

FAQs

No. It covers the EC2 instances only. DBU rates for Graviton node types may differ.