Skip to main content
A SageMaker notebook instance is billed for every hour it is running, whether or not anyone has a notebook open. nOps finds notebook instances that have run almost every hour for the last 30 days, weekends included, and estimates the saving from stopping them outside a 60-hour working week (12 hours on each weekday).

Why it matters

Notebook instances are often started for an experiment and never stopped. One left running nights and weekends costs close to three times what it would on a working-hours schedule.

How nOps finds it

nOps reads the last 30 complete days of SageMaker notebook instance billing. An instance is flagged when it was billed on average at least 20 hours a day, with at least 24 days of 20 or more hours, it has been billed since the start of the window and is still running. Instances mostly covered by Savings Plans are left out. The saving is the last 7 days of on-demand cost scaled to a month, times the share of the week outside 60 working hours (108 of 168 hours, about 64%). Savings under $5 a month are not shown.

Before you start

Permission to update notebook instances and create lifecycle configurations (sagemaker:UpdateNotebookInstance, sagemaker:CreateNotebookInstanceLifecycleConfig).

How to implement

  1. Ask the notebook owner whether anything runs in it overnight, such as training in the notebook kernel.
  2. Create a lifecycle configuration that runs the AWS auto-stop-idle script, which stops the instance after a set idle time (SageMaker console: Notebook → Lifecycle configurations → Create configuration).
  3. Stop the notebook instance, attach the lifecycle configuration (Edit → Lifecycle configuration), and start it again.
  4. If you prefer a fixed schedule, call aws sagemaker stop-notebook-instance from an EventBridge Scheduler rule in the evening.
  5. If the instance is defined in Terraform or CloudFormation, add the lifecycle configuration there too.

Risks and considerations

  • Attaching a lifecycle configuration needs a stop and start of the instance.
  • Work in progress that is not saved to the instance volume is lost when it stops. The ML storage volume is kept and still billed.
  • The estimate assumes a 60-hour working week. Teams in several time zones may need a longer schedule.
  • To roll back, detach the lifecycle configuration or remove the schedule.

FAQs

No. Files on the instance’s ML storage volume are kept. Only the compute stops being billed.
It stands for 12 hours on each weekday. If your team works longer hours, the saving is lower.