Why it matters
Long retention is usually set for compliance, and those old logs are rarely searched. Paying search-ready prices for them is the expensive way to keep them.How nOps finds it
nOps reads Cloud Logging retention charges for each log bucket. A log bucket is flagged when it has been billed for at least 7 days and in the last 2 days. The volume is worked out from the cost at the $0.01 per GiB-month retention list price. The saving is the retention cost over the last 7 days, scaled to a month, times 1 minus the Archive price over the retention price: Archive lists at $0.0024 per GiB-month for multi-region and global locations (a 76% saving) and $0.0012 for a single region (88%). It must be at least $5 a month. Projects whose spend on the service is at least 90% paid by credits or promotions are left out.Before you start
Confirm your compliance rules allow logs older than 30 days to be kept outside Cloud Logging, where they cannot be searched in Logs Explorer.How to implement
- Create a Cloud Storage bucket with the Archive storage class in a suitable location, with a retention policy if you need one.
- Create a log sink that routes the same logs to that bucket:
gcloud logging sinks create SINK_NAME storage.googleapis.com/BUCKET_NAME --log-filter='FILTER'. - Give the sink’s writer identity the Storage Object Creator role on the bucket.
- Once logs arrive in the bucket, set the log bucket’s retention to 30 days:
gcloud logging buckets update LOG_BUCKET --location=LOCATION --retention-days=30.
Risks and considerations
- A sink only routes new logs. Shortening retention deletes logs older than 30 days from Cloud Logging, so export them first if you need them.
- Logs in Cloud Storage cannot be searched in Logs Explorer. Archive reads cost extra and Archive objects have a 365-day minimum storage charge.
- To roll back, set the log bucket’s retention back to its previous value. Logs already deleted from Cloud Logging do not come back.
FAQs
Do log sinks cost anything?
Do log sinks cost anything?
Routing logs with a sink is free. You pay for the Cloud Storage bucket that receives them.