Lambda Function to dynamically update content from S3 to EC2

In this blog, we will design and deploy a robust infrastructure to host a web application using Nginx on two EC2 instances managed by an Auto Scaling Group (ASG). The setup will include a Load Balancer for high availability, data retrieval from an S3 bucket, and a Lambda function to dynamically update the index.html file on the EC2 instances whenever a new version is uploaded to the S3 bucket.

Step 1: Setting Up the S3 Bucket

Create an S3 bucket:
- Navigate to the S3 console and create a bucket.
- Enable versioning to keep track of changes.

Upload the initial index.html file:

Upload your index.html file to the bucket under the public/ folder (e.g., public/index.html).

You can create your own index.html file or use the below html content.

  <!DOCTYPE html>
  <html lang="en">
  <head>
      <meta charset="UTF-8">
      <meta name="viewport" content="width=device-width, initial-scale=1.0">
      <title>Travel Information</title>
      <style>
          body {
              font-family: Arial, sans-serif;
              margin: 0;
              padding: 0;
              background-color: #f4f4f9;
          }
          header {
              background-color: #4CAF50;
              color: white;
              text-align: center;
              padding: 1em;
          }
          nav {
              background: #333;
              color: white;
              display: flex;
              justify-content: space-around;
              padding: 0.5em;
          }
          nav a {
              color: white;
              text-decoration: none;
              padding: 0.5em;
          }
          nav a:hover {
              background: #575757;
          }
          section {
              margin: 2em auto;
              padding: 1em;
              max-width: 800px;
              background: white;
              border-radius: 5px;
              box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
          }
          footer {
              text-align: center;
              padding: 1em;
              background: #333;
              color: white;
              position: fixed;
              bottom: 0;
              width: 100%;
          }
      </style>
  </head>
  <body>
  <header>
      <h1>Welcome to Unoff Travels</h1>
      <p>Your ultimate guide to exploring the world</p>
  </header>
  <nav>
      <a href="#destinations">Destinations</a>
      <a href="#guides">Travel Guides</a>
      <a href="#contact">Contact Us</a>
  </nav>
  <section id="destinations">
      <h2>Top Destinations</h2>
      <ul>
          <li>Paris, France - The City of Love</li>
          <li>Kyoto, Japan - The Land of Temples</li>
          <li>Cape Town, South Africa - A Natural Paradise</li>
      </ul>
  </section>
  <section id="guides">
      <h2>Travel Guides</h2>
      <p>Explore our comprehensive guides to plan your next adventure.</p>
  </section>
  <footer>
      <p>&copy; 2024 Unoff Travels. All Rights Reserved.</p>
  </footer>
  </body>
  </html>

You can create a folder in s3 called public. Then upload your index.html file.

Set bucket policies:
- Allow read access for EC2 instances to fetch the index.html file.

Step 2: Configuring IAM Role for EC2 Instances

Go to IAM and create a role for EC2 service.
Attach AmazonS3FullAccess policy to the role.

We might need other policies in the future like AmazonSSMManagedInstanceCore, but for now, this will do.

Step 3: Configuring EC2 Instances and Auto Scaling Group

Create an EC2 Launch Template:
- Use the Amazon Linux 2 AMI.
  
  Choose instance type as t2.micro and create a new security group. Add the role created above in IAM instance profile section.
- Install Nginx during the instance initialization via the User Data script:

    #!/bin/bash
    yum update -y
    amazon-linux-extras enable nginx1
    yum install nginx -y
    systemctl start nginx
    systemctl enable nginx

Click on Create Launch Template.

Create an Auto Scaling Group (ASG):
- Attach the Launch Template to the ASG.

Choose the Network Configurations

Configure the ASG to maintain a minimum of 2 instances.

Similarly you can configure additional settings according to your requirement.

Attach a Load Balancer (ALB):
- Create an Application Load Balancer and attach it to the ASG.
- Configure the Target Group to use EC2 instances.
  Create a target group pointing to instances and don’t define any targets yet. Use the same target group for loadbalancer listener.
  
  Go to the austoscaling group you created before and click on Integrations.
  
  Edit Load balancing, select Application, Network or Gateway Load Balancer target groups.
  
  Click on Update.
- Check the targets under target group to see if they are in a healthy state:

You can also check if the default nginx webpage is visible by opening the loadbalancer DNS in your browser.

Step 4: Lambda Function to Update the `index.html`

Create the Lambda Function:

Navigate to the Lambda console and create a function.

Use Python runtime. Below is a sample Python script:

  import boto3

  s3 = boto3.client('s3')
  ec2 = boto3.client('ec2')
  ssm = boto3.client('ssm')

  def lambda_handler(event, context):
      bucket = "unoff-travel"
      key = event['Records'][0]['s3']['object']['key']

      if key != "public/index.html":
          return {"message": "File ignored"}

      # Download file from S3
      local_file_path = '/tmp/index.html'
      s3.download_file(bucket, key, local_file_path)

      # Get EC2 instances with specific tag
      response = ec2.describe_instances(Filters=[
          {'Name': 'tag:Name', 'Values': ['My-ASG-Instances']},
          {'Name': 'instance-state-name', 'Values': ['running']}
      ])

      instance_ids = [
          instance['InstanceId']
          for reservation in response['Reservations']
          for instance in reservation['Instances']
      ]

      if not instance_ids:
          return {"message": "No instances found"}

      # Upload file to EC2 instances using SSM
      with open(local_file_path, 'r') as file:
          content = file.read()

      commands = [
          "echo '{}' > /usr/share/nginx/html/index.html".format(content.replace("'", "\\'"))
      ]

      ssm_response = ssm.send_command(
          InstanceIds=instance_ids,
          DocumentName="AWS-RunShellScript",
          Parameters={'commands': commands}
      )

      command_id = ssm_response['Command']['CommandId']

      # Wait for the command to complete
      for instance_id in instance_ids:
          waiter = ssm.get_waiter('command_executed')
          waiter.wait(CommandId=command_id, InstanceId=instance_id)

      return {"message": "Index.html updated successfully on all instances"}

Set Permissions

Attach an IAM Role to Lambda with S3 read, SSM Full access, Cloudwatch Full Access and EC2 Full Acess permissions.
Set S3 trigger

Click on Add trigger

Select S3 and choose your bucket.

Click on the I acknowledge checkbox and Add.

Step 5: Testing the Setup

Verify EC2 Instances:
- Access the Load Balancer’s public DNS and check if the Nginx default page is loading.
Test File Update Workflow:
- Upload a new index.html file to the S3 bucket under the public/ folder.
- The Lambda function should trigger and update the file on all EC2 instances.
- Refresh the Load Balancer’s DNS to see the updated content.
  
  The content has been updated in the EC2 instances successfully.

Step 6: Monitoring and Logging

Enable CloudWatch Logs:
- Monitor the Lambda execution logs.
- Track S3 and Load Balancer access logs for debugging.
Set Alarms:
- Configure CloudWatch Alarms for ASG instance health and Lambda errors.

Creating a Lambda Function to dynamically update content from S3 to EC2