Introduction to Compute Copilot for Autoscaling Groups (ASG)

About Compute Copilot for ASG and how it works

• Compute Copilot is a service that automatically optimizes any compute-based workload. It leverages ML algorithms to reduce your AWS ASG costs by automatically migrating EC2 instances to Spot. 
• Compute Copilot Lambda is aware whenever ASG launches a new on-demand instance (such as in response to a desired capacity change). In response, it automatically launches a Spot instance with settings that mirror those of the on-demand instance, attaches it to the ASG, confirms it is in service, and removes the on-demand instance for you.

Why Compute Copilot

• Compute Copilot uses AI-driven decision making to provision and run auto scaling group instances at the cheapest price in real time, without manual effort.
• Compute Copilot allows you to benefit from Spot savings with the same reliability as on-demand. By analyzing historical data and Spot Termination events, it ensures your critical workloads remain safe from interruption. 
• Compute Copilot does not require your workload to be transferred to a proprietary system, but works directly with AWS ASG.
• Compute Copilot does not typically modify the ASG settings, allowing it to be disabled at any time without interrupting existing workflows.

Prerequisites

1. You must be logged in to your nOps account. 
2. Your AWS account must be configured to your nOps account.
3. You must have the correct permissions. 

In the Compute Copilot Auto Scaling Groups Onboarding call, we will launch a CloudFormation Stack to create (1) a Lambda and its associated resources, and (2) a StackSet to create an EventBridge Forwarder in the same account but in all the SCP enabled regions of the same account.

Two primary roles are needed to create this CloudFormation Stack:

1. AWSCloudFormationStackSetAdministrationRole
2. AWSCloudFormationStackSetExecutionRole

These roles should exist in the AWS account where most of your ASGs are present, with these exact names. Before the Onboarding call, we ask that you verify that the above roles exist to ensure we can proceed successfully. 

• If these roles exist, no further action is required prior to the Onboarding call. 

• If these roles do not exist, please follow the below steps involving IAM Roles and Policy Creation:

1. In the AWS account, create a Policy with the name “AssumeRole-AWSCloudFormationStackSetExecutionRole” as following:

{
"Version": "2012-10-17",
"Statement": [
	{
	  "Action": [
			"sts:AssumeRole"
	],
	  "Resource": [
			"arn:*:iam::*:role/AWSCloudFormationStackSetExecutionRole"
	],
	  "Effect": "Allow"
	}
  ]
}

1. Create a new IAM Role with the name “AWSCloudFormationStackSetAdministrationRole” and attach the Policy we created in the above step (“AssumeRole-AWSCloudFormationStackSetExecutionRole”). Once it is attached, create the IAM Role.
2. Navigate to the IAM Role we created →  Trust Relationships tab. Edit the trust relationship to the following:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudformation.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

1. Navigate back to IAM Roles, and create another new IAM Role named “AWSCloudFormationStackSetExecutionRole.” and for the policies we will be giving “AdministratorAccess” Permission Policy and let’s go ahead and create the role
   
   5. Navigate to Trust Relationships tab of the “AWSCloudFormationStackSetExecutionRole” and edit the trust relationship to the following:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::accountnumber:root"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

In the above Principal, the account number should be that of the account you are currently logged in to.

We have now successfully created the roles and policies necessary to create StackSets in this AWS Account. 

For further reference, please refer to AWS Documentation

Steps to Configure Your ASG Cluster

Generate API Key

• Navigate to Organization Settings → API key.

• Generate a New API Key, and save it to use later in the ASG Lambda CloudFormation stack

  Note: you can use one key for multiple AWS accounts or stacks.

Launch ASG Lambda Stack in the AWS Account

1. In the nOps dashboard, Navigate to Organization Settings → Integrations → ASG Lambda.
   • Select the AWS account where you want to run the ASG Lambda Stack.
   • Select Launch ASG Lambda Stack. You will be redirected to the Cloud formation stack in the AWS console.
   • Input the API key that you created in step 1.2 into the Token field.
   • Acknowledge the capabilities and create the stack.
   • Please verify in your AWS account that you are able to see the following StackSet and stack that was created via Terraform.

   Under Stacks, you should be able to see the above in Stack instances in the same region as stacks.
   Notes: Under StackSets, you should be able to see the above in Stackset in the same region as stacks. Please verify the status. You might see the stackset creating a stack only in few regions instead of all regions and this is not of concern as stackset fails to create a stack in other regions due to SCP’s

2. After successful completion, return to the nOps platform and refresh the ASG Lambda page.

3. You should see the updated version of Lambda, with the status showing a successful connection to the configured AWS account.

   

   Note: Before the ASG Lambda stack runs,

   • The status of the version shows N/A by default.
   • Auto Update is enabled by default, allowing Compute Copilot to automatically update to the newest version of Lambda. This setting can be disabled by the user if desired.

Select the ASG cluster and configure

1. Navigate to the ASG Dashboard and select the ASG cluster to configure
   • Open Configure Model. The details section of Auto Scaling Group will be prefilled. 
   • The AWS Lambda configuration section will show the version detail and current status of the Lambda Stack to confirm it is properly configured on the AWS account. 
   • You can create or choose an existing ASG template in the Spot detail section.
2. Create an ASG Template.
   • Give the ASG template a unique name
   • Choose the instance families, vCPU, and Memory suitable for your workload. From this pool, Compute Copilot will select the most optimal choice for price and stability.
   • It is recommended to select as many instance families as possible, to provide Compute Copilot with a wider recommendations pool

   Note: you can also clone an existing template to fetch pre-defined configurations to apply to a new template. 

3. Set the Spot percentage and Max Spot Instances using the draggable bars. 

Spot percentage defines the percentage of on-demand instances to be replaced with Spot. 

Max Spot Instances defines the maximum of Spot instances to be created by Compute Copilot.

1. Once values are defined, select Configure.  

2. ASG status will now display Configured.

What to expect after configuring ASG

Compute Copilot Lambda will begin replacing on-demand instances in this ASG with Spot alternatives, either every 30 minutes or upon the launch of a new on-demand instance.

FAQ

1. How does Compute Copilot replace on-demand instances?

   1. ASG launches a new on-demand instance
   2. Lambda intercepts the EC2 Instance State-change Notification event from EventBridge
   3. If the created instance is not protected from termination and should be replaced, Compute Copilot performs the following steps.
      1. Copy the Launch Template or Launch Configuration from the ASG launch template
      2. In the copied Launch Template, modify Network Interfaces, Tags, Block Device and/or Mappings from the instance Launch Template or Configuration if needed
      3. Fetch recommended instance types from nOps API
      4. Request Spot Fleet with the copied Launch Template and recommended instance types
      5. Once the Spot request is fulfilled, get the Spot Instance and wait for it to its state to be Running
      6. Attach the created Spot Instance to the ASG
      7. Wait for the attached Spot Instance’s state to be InService
      8. Terminate the on-demand instance

2. Why don’t I see some of the instance families when I create a Launch Template?

   Instance families that are not compliant with the ASG you are attempting to configure are automatically hidden. For example, if your ASG uses instances with arm64 CPU architecture, other instance families that are x86_64 will be filtered out from the list.

3. What is Auto Update Lambda, and how does it work? 

   When Auto Update is enabled, your Lambda will be automatically updated every time a new Compute Copilot version is released (12 p.m. UTC). When you deploy Compute Copilot Lambda with Auto Update, the stack will also contain a Cross-Account role.  

4. What permissions does Auto-Update Cross-Account policy have?

Action:
- cloudformation:CreateChangeSet
- cloudformation:DescribeStackInstance
- cloudformation:DescribeStackResources
- cloudformation:DescribeStacks
- cloudformation:GetTemplateSummary
- cloudformation:UpdateStack
Resource:
- !Ref AWS::StackId

Action:
- cloudformation:CreateChangeSet
- cloudformation:DescribeStackEvents
- cloudformation:DescribeStackInstance
- cloudformation:DescribeStackResources
- cloudformation:DescribeStacks
- cloudformation:DescribeStackSet
- cloudformation:DescribeStackSetOperation
- cloudformation:GetTemplateSummary
- cloudformation:UpdateStackInstances
- cloudformation:UpdateStackSet
Resource:
- !Sub NASGEventBridgeForwarderMultiRegion