Automating Amazon Linux Instance Creation using Terraform and Pre-Installed AMI – Part 2
In part 1 of this series, we learned about the process of creating Amazon Linux instances with pre-installed packages using Terraform and converting them into an Amazon Machine Image (AMI) for future deployments. This process helps to automate instance creation, saves time, and reduces manual effort. Additionally, using Terraform allows version control and easy maintenance of infrastructure as code. We also learned about the prerequisites for AWS and Terraform, how to install Terraform and AWS CLI, and how to structure the Terraform project by splitting the configuration code into multiple files.
Setup AWS credentials
In order to interact with AWS services through Terraform, you need to provide AWS access and secret keys to Terraform. These keys are used to authenticate with AWS and authorize Terraform to perform actions on your behalf.
Setting up AWS credentials in your Terraform code is important because it allows you to automate your infrastructure provisioning process without requiring manual intervention. By providing your credentials in your Terraform code, you can ensure that anyone who runs your code has the necessary permissions to provision or modify your infrastructure in AWS.
It is important to keep your AWS credentials secure and not expose them in public repositories or other insecure locations. One way to keep them secure is by using environment variables to store them or using AWS IAM roles with temporary credentials.
But in this blog, we will use static credentials. This is good for learning and testing purposes.
Note: Avoid using hard-coded credentials in production environments and never push them to code repositories.
Define AWS and Terraform providers
To authenticate with AWS, create authentication variables in variables.tf for AWS access key, AWS secret key, and the AWS region.
variables.tf
variable "aws_access_key" {
type = string
description = "AWS access key"
}
variable "aws_secret_key" {
type = string
description = "AWS secret key"
}
variable "aws_region" {
type = string
description = "AWS region"
}Now, define the Terraform providers for the AWS provider in main.tf and add the AWS credential information in terraform.tfvars for the access key, secret key, and region.
main.tf
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.0"
}
}
}
provider "aws" {
access_key = var.aws_access_key
secret_key = var.aws_secret_key
region = var.aws_region
}terraform.tfvars:
# AWS Settings
aws_access_key = "Your_access_key"
aws_secret_key = "Your_secret_key"
aws_region = "Your_region"Generate AWS Key Pair and Save .pem file in the current directory
It is required to create an AWS key pair to enable secure access to the instance via SSH.
main.tf# Generates a secure private key and encodes it as PEM
resource "tls_private_key" "key_pair" {
algorithm = "RSA"
rsa_bits = 4096
}
# Create the Key Pair
resource "aws_key_pair" "key_pair" {
key_name = "pemkey"
public_key = tls_private_key.key_pair.public_key_openssh
}
# Save file
resource "local_file" "ssh_key" {
filename = "${aws_key_pair.key_pair.key_name}.pem"
content = tls_private_key.key_pair.private_key_pem }Note: I've given a key_name of "pemkey" here, but you can use your own.
The above Terraform code will generate a secure RSA private key of 4096 bits and encode it as a PEM. It will then create an AWS key pair with the name “pemkey” and the public key generated from the private key. Finally, it saves the private key as a local file with the name “pemkey.pem”. This key pair will be used to access the EC2 instances that will be launched later in the code
Create VPC, subnet and other networking components
In this section, we will use Terraform to create a Virtual Private Cloud (VPC) on AWS and set up the necessary networking components like a public subnet, an internet gateway, and a route table. These networking components are crucial for creating and managing resources in the VPC. We will also assign a route table to the public subnet to allow resources to connect to the internet.
main.tf
# Create the VPC
resource "aws_vpc" "vpc" {
cidr_block = var.vpc_cidr
enable_dns_hostnames = true
}
# Define the public subnet
resource "aws_subnet" "public-subnet" {
vpc_id = aws_vpc.vpc.id
cidr_block = var.public_subnet_cidr
availability_zone = var.aws_az
}
# Define the internet gateway
resource "aws_internet_gateway" "gw" {
vpc_id = aws_vpc.vpc.id
}
# Define the public route table
resource "aws_route_table" "public-rt" {
vpc_id = aws_vpc.vpc.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.gw.id
}
}
# Assign the public route table to the public subnet
resource "aws_route_table_association" "public-rt-association" {
subnet_id = aws_subnet.public-subnet.id
route_table_id = aws_route_table.public-rt.id
}variables.tf
# AWS AZ
variable "aws_az" {
type = string
description = "AWS AZ"
default = "ap-south-1a"
}
# VPC Variables
variable "vpc_cidr" {
type = string
description = "CIDR for the VPC"
default = "10.3.0.0/16"
}
# Subnet Variables
variable "public_subnet_cidr" {
type = string
description = "CIDR for the public subnet"
default = "10.3.1.0/24"
}Note: You can replace the default values with your own values as per your needs.
It is not required to mention the variable values for AWS availability zone, VPC CIDR, and subnet CIDR in terraform.tfvars if you have already defined default values for those variables in variables.tf. If you don’t provide values for those variables in terraform.tfvars, Terraform will use the default values defined in variables.tf. However, if you want to override the default values, you can provide new values for those variables in terraform.tfvars.
terraform.tfvars
# Network
aws_az = "ap-south-1a"
vpc_cidr = "10.3.0.0/16"
public_subnet_cidr = "10.3.1.0/24"Create a security group
When creating an EC2 instance, it is important to set up a security group to control inbound and outbound traffic. In this section, we will define a security group using Terraform code with rules that allow incoming traffic on ports 80 and 8080 for HTTP traffic, port 22 for SSH connections, and outgoing traffic on all ports. We will also tag the security group for easy identification.
main.tf
# Define the security group for the EC2 Instance
resource "aws_security_group" "aws-vm-sg" {
name = "vm-sg"
description = "Allow incoming connections"
vpc_id = aws_vpc.vpc.id
ingress {
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "Allow incoming HTTP connections"
}
ingress {
from_port = 8080
to_port = 8080
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "Allow incoming connections with port 8080"
}
ingress {
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "Allow incoming SSH connections (Linux)"
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
tags = {
Name = "linux-sg"
}
}Note: Replace port 22 with 3389 to allow incoming RDP connections if you are going to create a Windows instance.
Create Operating System Version Variable
Defining the operating system version variables helps in maintaining the consistency of the infrastructure across different environments. In the below code snippet, the data block retrieves the latest Amazon Linux 2 AMI provided by Amazon and uses it to create EC2 instances with the same operating system. By defining the “os” variable, it becomes easier to change the operating system version in one place and propagate the changes throughout the code.
main.tf
# Get latest Amazon Linux 2 AMI
data "aws_ami" "amazon-linux-2" {
most_recent = true
owners = ["amazon"]
filter {
name = "name"
values = ["amzn2-ami-hvm*"]
}
}You can define your own operating system as per your requirements.
Here I have written a code snippet to get Amazon Linux 2 AMI, but if you want to get Ubuntu Linux, you can use the below code snippet in your Terraform configuration file.
# Get latest Ubuntu Linux Focal Fossa 20.04 AMI
data "aws_ami" "ubuntu-linux-2004" {
most_recent = true
owners = ["099720109477"] # Canonical
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-*"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
}Create an EC2 instance
This section of code snippet will create a Linux EC2 instance
main.tf
# Create EC2 Instance
resource "aws_instance" "vm-server" {
ami = data.aws_ami.amazon-linux-2.id
instance_type = var.vm_instance_type
subnet_id = aws_subnet.public-subnet.id
vpc_security_group_ids = [aws_security_group.aws-vm-sg.id]
source_dest_check = false
key_name = aws_key_pair.key_pair.key_name
associate_public_ip_address = var.vm_associate_public_ip_address
user_data = "${file("aws-user-data.sh")}"
# root disk
root_block_device {
volume_size = var.vm_root_volume_size
volume_type = var.vm_root_volume_type
delete_on_termination = true
encrypted = true
}
# extra disk
ebs_block_device {
device_name = "/dev/xvda"
volume_size = var.vm_data_volume_size
volume_type = var.vm_data_volume_type
encrypted = true
delete_on_termination = true
}
tags = {
Name = "linux-server-vm"
}
}The code creates an EC2 instance on AWS with the specified parameters, such as the Amazon Linux 2 AMI, the instance type, the security group, the subnet, and the key pair. It also sets up the root and extra disks and assigns tags to the instance. Additionally, the user_data field is set to the contents of the aws-user-data.sh file, which is located locally using the locals block. This file is used to customize the instance with the necessary software and configurations at launch time.
locals block in main.tf
locals {
user_data_path = "/path/to/aws-user-data.sh"
}Replace /path/to with the actual path on your machine where aws-user-data.sh file is located
variables.tf
variable "vm_instance_type" {
type = string
description = "EC2 instance type"
default = "t3.small"
}
variable "vm_associate_public_ip_address" {
type = bool
description = "Associate a public IP address to the EC2 instance"
default = true
}
variable "vm_root_volume_size" {
type = number
description = "Root Volume size of the EC2 Instance"
}
variable "vm_data_volume_size" {
type = number
description = "Data volume size of the EC2 Instance"
}
variable "vm_root_volume_type" {
type = string
description = "Root volume type of the EC2 Instance"
default = "gp2"
}
variable "vm_data_volume_type" {
type = string
description = "Data volume type of the EC2 Instance"
default = "gp2"
}
variable "vm_instance_name" {
type = string
description = "Data volume type of the EC2 Instance"
}terraform.tfvars
# Virtual Machine Settings
vm_instance_name = "sample-inst"
vm_instance_type = "t2.micro"
vm_associate_public_ip_address = true
vm_root_volume_size = 8
vm_root_volume_type = "gp2"
vm_data_volume_size = 10
vm_data_volume_type = "gp2"Create and associate Elastic IP to the EC2 instance [Optional]
In AWS, Elastic IP is used to assign a fixed IP address to an instance, which helps to ensure that the instance retains the same IP address even if it is stopped and started. This provides a reliable way to associate a specific IP address with an instance, which is useful in various scenarios, such as hosting websites or applications that require a consistent IP address for communication.
main.tf
# Create Elastic IP for the EC2 instance
resource "aws_eip" "vm-eip" {
vpc = true
tags = {
Name = "vm-eip"
}
}
# Associate Elastic IP to the EC2 Instance
resource "aws_eip_association" "vm-eip-association" {
instance_id = aws_instance.vm-server.id
allocation_id = aws_eip.vm-eip.id
}Create a Bash script to install docker and jenkins
The aws-user-data.sh file is used in this Terraform configuration to pass user data to the EC2 instance at launch time, which can be used to automate instance configuration tasks or run scripts.
aws-user-data.sh
#!/bin/bash
sudo yum update -y
sudo amazon-linux-extras install docker -y
sudo service docker start
sudo usermod -a -G docker $USER
sudo curl --silent --location http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo | sudo tee /etc/yum.repos.d/jenkins.repo
sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
sudo yum upgrade
sudo amazon-linux-extras install java-openjdk11 -y
sudo yum install jenkins -y
sudo systemctl enable jenkins
sudo systemctl start jenkins
sudo systemctl status jenkinsThis code is a bash script that installs and sets up Jenkins on an Amazon Linux 2 EC2 instance. It updates the operating system, installs Docker and Jenkins, configures Jenkins to start automatically, and starts the Jenkins service.
Define output values in Terraform
The output.tf file is used to define the output values that Terraform will display after executing the code.
output.tf
output "vm_server_instance_id" {
value = aws_instance.vm-server.id
}
output "vm_server_instance_public_dns" {
value = aws_instance.vm-server.public_dns
}
output "vm_server_instance_public_ip" {
value = aws_instance.vm-server.public_ip
}
output "vm_server_instance_private_ip" {
value = aws_instance.vm-server.private_ip
}Here the code is defining several output values related to the EC2 instance that was created. These outputs include the instance ID, public DNS, public IP, and private IP address of the EC2 instance. These output values can be useful for reference or as inputs to other resources or modules.
Commands to execute Terraform code from local machine
- terraform init: Initializes the working directory and downloads any required modules.
- terraform plan: Creates an execution plan that shows what changes Terraform will make to the infrastructure.
- terraform apply: Applies the changes specified in the execution plan to the infrastructure.
- terraform destroy: Destroys the Terraform-managed infrastructure.
Generate AMI from instance
After the EC2 instance has been successfully launched with the desired configuration, it’s time to follow the below procedure to create an AMI from it.
- Create an AMI from the instance:
- In the AWS Management Console, select the instance
- Right-click on the instance and select “Image”, then “Create Image”
- Fill in the details for the new AMI, such as the name and description
- Click “Create Image”
- Once the AMI has been created, you can launch new EC2 instances from it. To do this:
- Go to the EC2 dashboard in the AWS Management Console
- Click the “Launch Instance” button
- Choose the newly created AMI from the list of available AMIs
- Configure the instance as desired, then launch it
The new instance will be based on the AMI and will have all the pre-installed packages you specified.
Benefits of generating an AMI
Creating an AMI with pre-installed packages can be useful because it allows for faster and more efficient deployment of new instances with the desired software configuration. Instead of manually installing and configuring software on each new instance, the AMI can be used to launch new instances with the same software already installed. This saves time and effort and ensures consistency across multiple instances.
Final Thoughts
Automating Amazon Linux instance creation with Terraform and generating an AMI with pre-installed packages provides a streamlined approach to managing infrastructure on AWS.
By using code to define resources, infrastructure can be easily versioned, modified, and scaled up or down with minimal manual intervention.
Additionally, the use of pre-installed packages in the AMI ensures that new instances can be spun up quickly with all necessary software installed and configured, reducing the time and effort required to set up new servers. Overall, this approach offers a more efficient and reliable way to manage infrastructure on AWS.
Author Information
Srija Anaparthi is a DevOps Engineer with a passion for exploring the latest cloud technologies. An Azure and AWS enthusiast sharing insights and experiences through thought-provoking blogs
