Instance Ignition 1.2

Instance Ignition: Automated Cloud Control for Cost Efficiency

1. Overview

This project automates the start and stop of AWS EC2 instances at specific times to optimize resource utilization and reduce operational costs. By leveraging AWS CloudWatch Events (EventBridge) and AWS Lambda, the solution dynamically triggers instance actions based on a pre-defined schedule. The automation not only enhances cost efficiency but also simplifies resource management by ensuring that instances run only during required business hours (or the hours set by the user).

Purpose and Motivation

Cloud computing enables organizations to scale resources on demand. However, running EC2 instances continuously—especially when not needed—can lead to unnecessary expenses. Automation is critical in optimizing costs and ensuring that resources are allocated only when required.

The idea behind "Instance Ignition" is to address the challenge of cost inefficiencies due to idle EC2 instances. By automating the start/stop actions based on time, businesses can significantly cut down on wasted resources and ensure that their cloud environment operates in an optimized manner.

With Instance Ignition:

  • Automate EC2 instance operations using AWS Lambda and CloudWatch Events.
  • Develop a reliable schedule that triggers instance start and stop actions at specific times.
  • Enhance overall operational efficiency and reduce costs by ensuring instances run only when needed.

Architecture

The solution consists of the following AWS components:

  • AWS EC2: Hosts the virtual machines that need to be managed.
  • AWS Lambda: Executes the Python code to start/stop EC2 instances.
  • Amazon CloudWatch Events (EventBridge): Triggers the Lambda function based on a set schedule.
  • AWS IAM: Manages the necessary permissions for Lambda to control EC2 instances.

A high-level architecture diagram:

Architecture Diagram

How to Use

Prerequisites

  • AWS Account with permissions for S3, Lambda, IAM, CloudWatch, and EventBridge.
  • Familiarity with Python, AWS CLI, and basic knowledge of AWS services.

Step-by-step Setup:

  1. Create a New Role:
    • Navigate to the AWS IAM console and select “Create Role.”
    • Choose “AWS service” as the trusted entity and select “Lambda.”
  2. Attach Managed Policies:
    • Attach the existing role AWSLambdaBasicExecutionRole for logging permissions. Click ‘Next’.
    • Attach a custom policy with the following JSON by using the ‘Create Inline Policy’ option in the ‘Add Permissions’ drop-down menu. Copy and paste the following JSON:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:StartInstances",
        "ec2:StopInstances",
        "ec2:DescribeInstances"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    }
  ]
}

Give the custom role a name and create it.

Role Creation

4.2 Lambda Function Development

Steps:

  1. Create Lambda Function:
    • Go to the AWS Lambda console and click “Create function.”
    • Name the function (e.g., EC2Scheduler) and select “Python 3.x” as the runtime.
    • Click on “Change default execution role” and assign the IAM role you created earlier.
      • Lambda Setup

Lambda Function Code:

Copy and paste the following python code:

import boto3
from datetime import datetime, timezone

# Initialize the EC2 client
ec2_client = boto3.client('ec2')

# Define the target EC2 instance IDs
INSTANCE_IDS = ['i-0123456789abcdef0']  # Replace with your instance IDs

def lambda_handler(event, context):
    # Get the current UTC time
    now = datetime.now(timezone.utc)
    current_hour = now.hour

    # Define schedule times (UTC)
    start_hour = 14  # Start instance at 14:00 UTC
    stop_hour = 23   # Stop instance at 23:00 UTC

    print(f"Current UTC hour: {current_hour}")

    if current_hour == start_hour:
        # Start EC2 instance(s)
        response = ec2_client.start_instances(InstanceIds=INSTANCE_IDS)
        print(f"Starting instances: {INSTANCE_IDS}")
    elif current_hour == stop_hour:
        # Stop EC2 instance(s)
        response = ec2_client.stop_instances(InstanceIds=INSTANCE_IDS)
        print(f"Stopping instances: {INSTANCE_IDS}")
    else:
        print("No scheduled action for this hour.")

    return {
        'statusCode': 200,
        'body': 'Lambda executed successfully.'
    }
  1. Deploy the Function:

    Click “Deploy” to save the lambda function code.

    2. Deploy Lambda

4.3 CloudWatch EventBridge Setup

  1. Create a New Rule:
    • In the AWS Eventbridge console, navigate to “Rules” and click “Create rule.”
    • For the “Rule Type”, select “Schedule”
    • Use the following cron expression to trigger the Lambda Function at the beginning of every hour: cron(0 * ? * *).
    2. Create Rule
  2. Set Target:
    • Under “Targets,” choose “Lambda function” and select the EC2Scheduler function.
  3. Review and Enable:
    • Name the rule (e.g., EC2SchedulerRule), review the settings, and enable the rule.

Sample Rule JSON:

{
  "Name": "EC2SchedulerRule",
  "ScheduleExpression": "cron(0 * * * ? *)",
  "State": "ENABLED",
  "Targets": [
    {
      "Id": "Target0",
      "Arn": "arn:aws:lambda:us-east-1:123456789012:function:EC2Scheduler"
    }
  ]
}

Make sure to replace the ARN details with your specific region and account ID.

5. Testing and Validation

  1. Test Strategy:
    • Simulate Lambda Invocations:
      • Use the “Test” feature in the Lambda console to invoke the function manually.
      • Adjust the current time in the code temporarily (or simulate the event) to force start/stop actions.
    • CloudWatch Logs:
      • Check the CloudWatch Logs for your Lambda function.
      • Verify the printed messages confirm whether the start or stop command was executed.
  2. Results:
    • Successful Instance Control:
      • When the Lambda function is triggered at 14:00 UTC, EC2 instances should start.
      • When triggered at 23:00 UTC, the instances should stop.
    • Log Evidence:
      • CloudWatch Logs should show messages like “Starting instances: [...]” or “Stopping instances: [...]” based on the current hour.
  3. Troubleshooting:
    • No Action Executed:
      • Confirm the Lambda function’s logic by verifying the current UTC time.
      • Check IAM permissions if the EC2 start/stop API calls fail.
    • CloudWatch Events Not Triggering:
      • Ensure the rule is enabled and correctly configured with the cron expression.

6. Discussion and Analysis

  1. Success Metrics:
    • Automation: The Lambda function is successfully invoked at the scheduled times and executes the correct actions.
    • Cost Efficiency: EC2 instances are running only during business-critical hours, leading to a reduction in unnecessary costs.
    • Reliability: The system logs indicate consistent and reliable execution of scheduled tasks.
  2. Lessons Learned:
    • Time Zone Handling: It is critical to understand UTC vs. local time to ensure actions occur at the intended times.
    • IAM Security: Least privilege should be practiced; always narrow down permissions to only what is necessary.
  3. Potential Enhancements:
    • Dynamic Scheduling: Add a user interface (using AWS API Gateway and a simple web app) to allow users to set custom start/stop times.
    • Multiple Instance Support: Modify the Lambda code to read configurations from a data store (such as DynamoDB) for handling different schedules for multiple instances.

7. Conclusion

This project, "Instance Ignition: Automated Cloud Control for Cost Efficiency," successfully demonstrates how to automate the management of AWS EC2 instances using CloudWatch Events and Lambda. By executing scheduled actions based on a time-triggered schedule, the solution reduces operational costs and improves resource utilization. Future work can include dynamic scheduling and broader integration with other AWS services for enhanced automation.