Skip to main content
Flow logs delivered to CloudWatch Logs are billed as vended log ingestion. Delivering the same logs to S3 is priced lower per GB, and you can still query them with Athena. nOps finds log groups that receive flow logs and estimates the saving from changing the destination to S3.

Why it matters

Flow logs on busy VPCs produce a lot of data, and CloudWatch ingestion is often the biggest part of their cost.

How nOps finds it

nOps reads the last 7 complete days of CloudWatch vended-log ingestion for log groups whose names look like flow-log groups. It works out which public volume tier your per-GB price matches, then compares the CloudWatch price with the S3 delivery price for that tier. The saving is the ingestion cost times that price difference, scaled to a month, and must be at least $25 a month. The cost of storing the logs in S3 is not deducted, and the CloudWatch storage you stop paying for is not added.

Before you start

An S3 bucket for the logs, with a bucket policy that lets the log delivery service write to it, and permission to create and delete flow logs (ec2:CreateFlowLogs, ec2:DeleteFlowLogs).

How to implement

  1. Check whether anything uses the logs in CloudWatch: metric filters, subscription filters, alarms or dashboards.
  2. Create a new flow log for the same VPC, subnet or interface with Destination: S3 (aws ec2 create-flow-logs --log-destination-type s3 --log-destination arn:aws:s3:::BUCKET/PREFIX). Parquet format makes Athena queries cheaper.
  3. Check that logs arrive in S3, then delete the old flow log that sends to CloudWatch Logs.
  4. Set a retention period or lifecycle rule on both the old log group and the S3 prefix.
  5. Update Terraform or CloudFormation so the CloudWatch destination is not recreated.

Risks and considerations

  • Metric filters, subscription filters and alarms on the log group stop working. Rebuild them on S3 with Athena or another tool if needed.
  • S3 storage of the delivered logs is added. Set a lifecycle rule so old logs expire.
  • Capturing only rejected traffic instead of all traffic reduces volume further, if that is enough for your needs.
  • To roll back, create a flow log that sends to CloudWatch Logs again.

FAQs

No. You create a new flow log with the S3 destination and delete the old one.