> ## Documentation Index
> Fetch the complete documentation index at: https://help.nops.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Use Spot Node Pools for Non-Production GKE Clusters

> Move on-demand node pools in non-production GKE clusters to Spot VMs, which list 60-91% below on-demand.

GKE node pools can run on Spot VMs, which Compute Engine lists at 60% to 91% below on-demand but can reclaim at any time. Non-production clusters can usually handle that, because Kubernetes reschedules pods when a node goes away. nOps finds on-demand node pools whose cluster, node pool, node or project names mark them as non-production, and estimates the saving from moving them to Spot.

| | |
| - | - |
| **Cloud** | Google Cloud |
| **Category** | Compute |
| **Service** | Google Kubernetes Engine |
| **Action** | Modify |
| **Effort** | Medium |
| **Restart needed** | Yes |
| **Can be rolled back** | Yes |
| **Analysis window** | Last 7 days |
| **Source** | nOps |

## Why it matters

Development and test clusters often run on the same on-demand nodes as production. Spot node pools are one of the largest single savings available for those clusters.

## How nOps finds it

nOps reads on-demand Compute Engine vCPU and memory billing for GKE nodes, grouped by node pool, over the last 7 complete days. A node pool is flagged when it was billed on all 7 days and at least 90% of its cost carries a non-production word (such as dev, test, staging or sandbox) in its cluster, node pool, node or project name, with no production word. Pools with more than 10% GPU or sole-tenant cost, pools already mostly on Spot, and pools where committed-use discounts cover more than half the cost are left out. The saving assumes Spot costs 40% of the on-demand list price (the low end of Google's published 60-91% discount): it is the pool's current cost after credits minus 40% of its list cost, scaled to a month, and must be at least \$25 a month. Projects whose spend on the service is at least 90% paid by credits or promotions are left out.

## Before you start

Workloads in the cluster must tolerate a node being removed at short notice (several replicas, or jobs that can restart).

## How to implement

1. Check which workloads run in the node pool and that they can be restarted on another node.
2. Create a Spot node pool with the same machine type: `gcloud container node-pools create POOL_NAME --cluster=CLUSTER --spot --machine-type=MACHINE_TYPE --region=REGION` (or `--zone`).
3. Move workloads to it, using a node selector on `cloud.google.com/gke-spot=true` or a taint and toleration.
4. Keep a small on-demand node pool for system pods and anything that must not be interrupted.
5. Scale down and delete the old on-demand pool, and update Terraform so it is not recreated.

## Risks and considerations

* Spot capacity can be reclaimed at any time with about 30 seconds' notice, and may not be available in every zone at every moment.
* The real discount depends on the machine type and region, so the saving can be higher than the estimate.
* To roll back, move the workloads back to an on-demand node pool.

## FAQs

> [accordion] How does nOps decide a cluster is non-production?
>
> From words in the cluster, node pool, node and project names only, such as dev, test, staging, qa or sandbox. Labels are not used for this recommendation.

## Related

* [Use Spot VMs for Non-Production Compute Engine Instances](https://help.nops.io/inform/supported-recommendations/use-spot-vms-for-non-production-compute-engine-instances)
* [Schedule Non-Production Compute Engine Instances](https://help.nops.io/inform/supported-recommendations/schedule-non-production-compute-engine-instances)
* [Supported recommendations](https://help.nops.io/inform/supported-recommendations.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.