In the dynamic landscape of cloud computing, keeping a close watch on resource consumption is crucial for cost management, security, and performance optimization. AWS provides a suite of tools to manage and monitor resources, but automating these checks using Bash scripting can simplify this process. This blog will walk you through a step-by-step guide to create a script that tracks AWS resources and automates its execution using a cron job.
Why Shift to the Cloud?
Organizations are increasingly migrating to cloud infrastructures for several compelling reasons:
Cost Savings: Reduces capital expenditure on hardware and software.
Scalability: Easily scale resources up or down based on demand.
Disaster Recovery: Robust disaster recovery solutions.
Flexibility: Enables rapid experimentation and innovation.
Remote Access: Facilitates access from any location with internet.
Faster Deployment: Quicker deployment of applications and services.
Security: Enhanced security measures and compliance options.
Collaboration: Improved collaboration with cloud-based tools and services.
Project Overview: AWS Resource Tracker
This project involves creating a Bash script to track and report the usage of AWS resources including S3, EC2, Lambda, and IAM Users, and also automate the execution of the project using a cron job.
Step-by-Step Guide
1. Create an EC2 Instance
First, launch an EC2 instance running Ubuntu and connect to it using SSH.
2. Install AWS CLI
Install the AWS Command Line Interface (CLI) on your EC2 instance:
sudo apt update
sudo apt install awscli -y
3. Configure AWS CLI
Configure the AWS CLI with your access credentials:
aws configure
You will be prompted to enter your AWS Access Key, Secret Key, region, and output format which you can obtain from Security Credentials on the AWS Console.
4. Create the Bash Script
Create and edit the script using a text editor like vim:
vim aws_resource_tracker.sh
The script given below has some metadata and consists of commands that are responsible for tracking the resources:
#!/bin/bash
###############################
# Author : "Your Name"
# Date : "Mention the date here"
#
# This script is responsible for tracking your AWS Resources.
###############################
set -x # This will run the script in debug mode
echo "Listing all S3 buckets:"
aws s3 ls
echo "Listing all EC2 instances:"
aws ec2 describe-instances
echo "Listing all Lambda functions:"
aws lambda list-functions
echo "Listing all IAM users:"
aws iam list-users
5. Make the Script Executable
Grant execute permission to the script:
chmod u+x aws_resource_tracker.sh
6. Execute the Script
Run the script to check the output:
./aws_resource_tracker.sh
To make the output more readable, add some echo statements and use the jq parser to filter JSON outputs. Here’s the refined script:
#!/bin/bash
###############################
# Author : "Your Name"
# Date : "Mention the date here"
#
# This script is responsible for tracking your AWS Resources.
###############################
set -x # This will run the script in debug mode
echo "Listing all S3 buckets:"
aws s3 ls
echo "Listing all EC2 instances:"
aws ec2 describe-instances | jq '.Reservations[].Instances[].InstanceId'
echo "Listing all Lambda functions:"
aws lambda list-functions
echo "Listing all IAM users:"
aws iam list-users
7. Automate the Script Execution Using Cron Job
To automate the script execution daily (for example at 2 AM) use cron jobs. Edit the crontab file:
crontab -e
Add the following line to schedule the script:
0 2 * * * /path/to/aws_resource_tracker.sh
8. Conclusion
By following these steps, you have created a script that tracks AWS resources and automated its execution using cron. This script helps in monitoring resource consumption efficiently, ensuring better cost management and resource optimization.