Why it matters
The EC2 instances behind a Pro warehouse are easy to miss because they appear on the AWS bill, not the Databricks bill. Serverless also starts in seconds and stops quickly when idle, which can save more.How nOps finds it
nOps reads the last 7 complete days of classic and Pro SQL warehouse DBUs and the on-demand EC2 instances tagged with that warehouse. The serverless price is your own billed serverless SQL rate for the same tier and region over the last 30 days (at least 10 DBUs). The saving is the warehouse DBU cost plus its on-demand EC2 cost, minus the same DBUs at the serverless rate, scaled to a month, and must be at least $25 a month. Warehouses whose EC2 is mostly covered by Reserved Instances or Savings Plans, warehouses with no matched EC2, and workspaces with 90% or more of Databricks spend covered by credits are left out. The extra saving from serverless stopping sooner when idle is not counted.Before you start
Serverless SQL enabled for the workspace, and no network or instance-profile setup that serverless cannot use.How to implement
- Confirm serverless SQL is enabled for the workspace and supports the warehouse’s network setup (for example private connectivity to data sources).
- In Databricks, open SQL Warehouses, edit the warehouse and change its type to Serverless, or create a new serverless warehouse of the same size.
- Set auto stop to a few minutes.
- Point dashboards, queries and BI tools at the warehouse (the ID stays the same if you edit it in place).
- Check that the EC2 instances tagged with the warehouse stop appearing in your AWS bill.
Risks and considerations
- Changing the warehouse type restarts it, so running queries are interrupted.
- Serverless warehouses run in the Databricks account, so instance profiles and some network setups need serverless equivalents.
- To roll back, change the type back to Pro or classic.
FAQs
Why can serverless be cheaper when its DBU rate is higher?
Why can serverless be cheaper when its DBU rate is higher?
The serverless rate includes the compute. With Pro or classic you pay the DBU rate plus the EC2 instances in your own AWS account.