Project Summary
This project demonstrates a multi-environment IaC (Infrastructure as Code) setup using Terraform and Ansible. It manages multiple environments—Dev, Staging, and Production—ensuring seamless deployment, scaling, and resource provisioning across all environments. The architecture integrates key cloud services like compute, storage, and security keys, leveraging automation tools for efficiency and consistency.
Step-by-Step Guide:
1. Setting Up the Ubuntu OS Environment
To install Terraform and Ansible, you need access to an Ubuntu 20.04 or 22.04 LTS environment. This environment can be:
A local machine running Ubuntu OS.
An AWS EC2 instance with Ubuntu as the operating system.
A virtual machine (e.g., VirtualBox or VMware) running Ubuntu.
Provisioning an Ubuntu EC2 Instance (Optional)
If using AWS, follow these steps to set up an Ubuntu EC2 instance:
Log in to the AWS Management Console.
Navigate to EC2 > Launch Instances.
Choose an Ubuntu AMI (e.g., Ubuntu 20.04 LTS).
Select an instance type (e.g., t2.micro for free tier eligibility).
Configure security groups to allow SSH, HTTP, and HTTPS access (ports 22, 80, 443).
Launch the instance and connect to it using the following command:
ssh -i your-key.pem ubuntu@your-ec2-public-ip
2. AWS CLI
Install and configure the AWS CLI to interact with AWS services.
To install AWS CLI on Ubuntu, run:
sudo apt-get update sudo apt-get install awscli -y aws --versionConfigure it using:
aws configureEnter your Access Key ID, Secret Access Key, region, and output format when prompted.
3. Access Keys and Permissions
Obtain AWS IAM Access Keys (Access Key ID and Secret Access Key) with appropriate permissions for Terraform to provision infrastructure.
Ensure the IAM user or role has access to EC2, S3, and IAM resources.
4. AMI Information
Have the AMI ID of the operating system image you plan to use.
You can find this in the AWS Management Console or via AWS CLI:
aws ec2 describe-images --filters "Name=name,Values=your-ami-name"
1. Installing Terraform and Ansible
a. Installing Terraform on Ubuntu
Follow these steps to install Terraform on Ubuntu:
Update the Package List
sudo apt-get updateInstall Dependencies
sudo apt-get install -y gnupg software-properties-commonAdd HashiCorp's GPG Key
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpgAdd the HashiCorp Repository
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.listInstall Terraform
sudo apt-get update && sudo apt-get install terraformVerify the Installation
terraform --version
b. Installing Ansible on Ubuntu
Ansible simplifies configuration management and automation. To install it:
Add the Ansible PPA
sudo apt-add-repository ppa:ansible/ansibleUpdate the Package List
sudo apt updateInstall Ansible
sudo apt install ansibleVerify the Installation
ansible --version
2. Creating Directories for Terraform and Ansible
To keep your infrastructure code and server configuration scripts organized, create two separate directories: one for Terraform and another for Ansible.
Navigate to Your Project Directory (or create a new one):
mkdir <your-project-name> && cd <your-project-name>Create a Directory for Terraform:
mkdir terraformCreate a Directory for Ansible:
mkdir ansibleVerify the Directory Structure:
treeYour project structure should look like this:
<your-project-name>/ ├── terraform/ └── ansible/
With this structure, you can separate your Terraform scripts (infrastructure provisioning) and Ansible playbooks (server configuration) efficiently.
3. Setting Up Infrastructure Directory in Terraform (With File Content)
After creating the infra directory, add basic configurations to each Terraform file to provision essential AWS resources.
Steps to Create the Infrastructure Directory and Add File Content
Navigate to the Terraform Directory:
cd terraformCreate the
infraDirectory:mkdir modules && cd modulesCreate and Populate the Terraform Files: below is code which i have used to create infrastructure structure to accomplish project pattern
Generate SSH Keys (
tws-terra-key-pandtws-terra-key-p.pub)note : here I have used key name as tws-terra-key-p , you can create with any name , and replace that every-where that old one appears,
To create SSH keys for accessing the EC2 instances, use the
ssh-keygencommand:ssh-keygen -t rsa -b 2048 -f tws-terra-key-p -N ""
Final Directory Structure
At this point, your Terraform project structure should look like this:
├── tws-terra-key-p # Private SSH key for EC2 access
├── tws-terra-key-p.pub # Public SSH key for EC2 access
├── modules
│ ├── mybucket.tf
│ ├── my_dynamodb.tf
│ ├── my_ec2.tf
│ ├── output.tf
│ └── variables.tf
├── main.tf # Defines environment-based modules
├── outputs.tf # AWS provider configuration
├── providers.tf # Backend configuration for state management
├── terraform.tf
Next Steps
Run Terraform Commands
Run the following commands to initialize, plan, and apply your Terraform setup:
a. terraform init : Initialize Terraform with the required providers and modules
b. terraform plan : Review the plan to apply changes
c. terraform apply : Apply the changes to provision infrastructure
Instances :
Buckets :
DynamoDb tables:
Secure the Private Key
Before using the private key, ensure that it is securely encrypted by setting proper permissions. This prevents other users from accessing it. Run the following command to restrict the access:
chmod 400 tws-terra-key-p # Set read-only permissions for the owner to ensure security
This command ensures that the private key (tws-terra-key-p) is only readable by you, preventing others from accessing or modifying it.
Access EC2 Instances
After provisioning, you can SSH into the EC2 instances created by terraform using the generated tws-terra-key-p:
ssh -i tws-terra-key-p ubuntu@<your-ec2-ip>
Terraform steps done ,now going to setup with ansible
5. Creating dynamic inventories in ansible dir
Firstly navigate to ansible dir which would you have created before
Step 1: Create the Inventories Directory
mkdir -p inventories/dev inventories/prod inventories/stg
Step 2: Add Inventory Content for each Environment
For inventories/dev: You need to provide the public IPs of the Dev infrastructure, which you can retrieve from the Terraform outputs.
[servers]
server1 ansible_host=3.249.218.238
server2 ansible_host=34.241.195.105
[servers:vars]
ansible_user=ubuntu
ansible_ssh_private_key_file=~/.ssh/tws-terra-key-p
ansible_python_interpreter=/usr/bin/python3
For inventories/stg:
[servers]
server1 ansible_host=34.244.89.121
server2 ansible_host=34.242.151.189
[servers:vars]
ansible_user=ubuntu
ansible_ssh_private_key_file=~/.ssh/tws-terra-key-p
ansible_python_interpreter=/usr/bin/python3
For inventories/prod:
[servers]
server1 ansible_host=3.252.144.3
server2 ansible_host=63.34.12.124
server3 ansible_host=34.244.48.139
[servers:vars]
ansible_user=ubuntu
ansible_ssh_private_key_file=~/.ssh/tws-terra-key-p
ansible_python_interpreter=/usr/bin/python3
Resulting Directory Structure
inventories
├── dev
├── prod
└── stg
6. Creating playbook for installing Nginx on all servers
Step 1: Navigate to the Ansible Directory
If you're not already in the Ansible directory, navigate to it first:
cd ../ansible
Step 2: Create the playbooks Directory
Create the playbooks directory inside the Ansible directory:
mkdir playbooks
Step 3: Navigate to the playbooks Directory
Now, navigate into the playbooks directory:
cd playbooks
Step 4: Create the install_nginx_playbook.yml File
Create the install_nginx_playbook.yml file with the following content to install Nginx and render a webpage using the nginx-role:
---
- name: Install Nginx and render a webpage to it
hosts: servers
become: yes
roles:
- nginx-role
Step 5: Verify the Directory Structure
After completing the above steps, your Ansible directory structure should look like this:
ansible
├── inventories
│ ├── dev
│ ├── prod
│ └── stg
├── playbooks
│ └── install_nginx_playbook.yml
7. Now initializing Roles for nginx named nginx-role from ansible galaxy
Here are the steps to initialize the nginx-role using Ansible Galaxy, which will generate the necessary folder structure for managing all tasks, files, handlers, templates, and variables related to the Nginx role.
Step 1: Navigate to the playbooks Directory
If you're not already in the playbooks directory, navigate to it:
cd ansible/playbooks
Step 2: Initialize the nginx-role Using Ansible Galaxy
Now, use the ansible-galaxy command to initialize the nginx-role:
ansible-galaxy role init nginx-role
This will create the following directory structure within the nginx-role folder:
nginx-role
├── README.md
├── defaults
│ └── main.yml
├── files
│ └── index.html
├── handlers
│ └── main.yml
├── meta
│ └── main.yml
├── tasks
│ └── main.yml
├── templates
├── tests
│ ├── inventory
│ └── test.yml
└── vars
└── main.yml
Step 3: Add Custom Tasks and Files to Your nginx-role
Now that your role structure is ready, you can add your custom tasks and files.
3.1: Add tasks/main.yml
Create a tasks/main.yml file under the nginx-role/tasks/ directory. This file will contain all the steps to install, configure, and manage the Nginx service. Here's the content for your tasks/main.yml:
---
# tasks file for nginx-role
- name: Install nginx
apt:
name: nginx
state: latest
- name: Enable nginx
service:
name: nginx
enabled: yes
- name: Deploy webpage
copy:
src: index.html
dest: /var/www/html
This will ensure that:
Nginx is installed with the latest version.
Nginx service is enabled and starts automatically.
The
index.htmlfile is copied to the/var/www/htmldirectory, which is where the default Nginx webpage is served from.
3.2: Add a Custom index.html File
You can add an index.html file under the nginx-role/files/ directory. This file can be customized as per your needs. Here's a simplified version of the index.html file you provided, with basic content:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Welcome to My Terraform & Ansible Test Page</title>
<style>
body {
font-family: Arial, sans-serif;
text-align: center;
margin: 50px;
}
h1 {
color: #2c3e50;
}
p {
color: #34495e;
}
</style>
</head>
<body>
<h1>🚀 Terraform & Ansible Deployment Test</h1>
<p>Congratulations! This page was deployed using Terraform and Ansible.</p>
<p>Server IP: <span id="server-ip">Fetching...</span></p>
<script>
fetch('https://api64.ipify.org?format=json')
.then(response => response.json())
.then(data => {
document.getElementById('server-ip').innerText = data.ip;
})
.catch(error => console.error('Error fetching IP:', error));
</script>
</body>
</html>
Note: You can replace this HTML content with your own custom webpage content as needed. The goal here is to serve a simple webpage as part of the Nginx configuration.
8. To add the update_inventories.sh script to your Ansible directory and integrate it with your existing setup, follow these steps:
Step 1: Create the update_inventories.sh Script
In your ansible directory, create a new file named update_inventories.sh with the following content. This script will dynamically update the inventory files for dev, stg, and prod environments based on the IPs fetched from the Terraform outputs.
#!/bin/bash
# Paths and Variables
TERRAFORM_OUTPUT_DIR="/mnt/c/Users/dci-student/Desktop/terra-ansible-multi-env/terraform" # Replace with the actual Terraform directory path
ANSIBLE_INVENTORY_DIR="/mnt/c/Users/dci-student/Desktop/terra-ansible-multi-env/ansible/inventories" #Replace according to the inventory path
# Navigate to the Terraform directory
cd "$TERRAFORM_OUTPUT_DIR" || { echo "Terraform directory not found"; exit 1; }
# Fetch IPs from Terraform outputs
DEV_IPS=$(terraform output -json dev_infra_ec2_public_ips | jq -r '.[]')
STG_IPS=$(terraform output -json stg_infra_ec2_public_ips | jq -r '.[]')
PROD_IPS=$(terraform output -json prod_infra_ec2_public_ips | jq -r '.[]')
# Function to update inventory file
update_inventory_file() {
local ips="$1"
local inventory_file="$2"
local env="$3"
# Create or clear the inventory file
> "$inventory_file"
# Write the inventory header
echo "[servers]" >> "$inventory_file"
# Add dynamic hosts based on IPs
local count=1
for ip in $ips; do
echo "server${count} ansible_host=$ip" >> "$inventory_file"
count=$((count + 1))
done
# Add common variables
echo "" >> "$inventory_file"
echo "[servers:vars]" >> "$inventory_file"
echo "ansible_user=ubuntu" >> "$inventory_file"
echo "ansible_ssh_private_key_file=~/.ssh/tws-terra-key-p" >> "$inventory_file"
echo "ansible_python_interpreter=/usr/bin/python3" >> "$inventory_file"
echo "Updated $env inventory: $inventory_file"
}
# Update each inventory file
update_inventory_file "$DEV_IPS" "$ANSIBLE_INVENTORY_DIR/dev" "dev"
update_inventory_file "$STG_IPS" "$ANSIBLE_INVENTORY_DIR/stg" "stg"
update_inventory_file "$PROD_IPS" "$ANSIBLE_INVENTORY_DIR/prod" "prod"
echo "All inventory files updated successfully!"
This script will:
Navigate to the Terraform directory and fetch the public IPs of the instances for dev, stg, and prod environments.
Dynamically generate or update the corresponding inventory files in the
ansible/inventoriesdirectory.Add common variables for all servers in each environment's inventory file.
Step 2: Verify the Directory Structure
After adding the script, your ansible directory should look like this:
ansible
├── inventories
│ ├── dev
│ ├── prod
│ └── stg
├── playbooks
│ ├── install_nginx_playbook.yml
│ └── nginx-role
│ ├── README.md
│ ├── defaults
│ │ └── main.yml
│ ├── files
│ │ └── index.html
│ ├── handlers
│ │ └── main.yml
│ ├── meta
│ │ └── main.yml
│ ├── tasks
│ │ └── main.yml
│ ├── templates
│ ├── tests
│ │ ├── inventory
│ │ └── test.yml
│ └── vars
│ └── main.yml
├── update_inventories.sh
Step 3: Make the Script Executable
Before running the update_inventories.sh script, ensure that it is executable. You can do this by running the following command:
chmod +x update_inventories.sh
Step 4: Run the Script
You can now execute the script to update the inventory files with the IPs fetched from Terraform:
./update_inventories.sh
Step 5: Verify the Inventory Files
After running the script, check the inventories directory. The dev, stg, and prod inventory files should now be updated with the IPs of your servers and the necessary variables.
Example contents of the dev inventory file:
[servers]
server1 ansible_host=192.168.1.10
server2 ansible_host=192.168.1.11
[servers:vars]
ansible_user=ubuntu
ansible_ssh_private_key_file=/home/amitabh/devops-key
ansible_python_interpreter=/usr/bin/python3
Repeat this process for stg and prod environments as well.
Step 6: Use the Updated Inventory in Playbooks
Now that your inventory files are updated, you can reference them in your Ansible playbooks by using the -i option:
- For dev inventory :
ansible-playbook -i inventories/dev playbooks/install_nginx_playbook.yml
- For stg inventory :
ansible-playbook -i inventories/stg playbooks/install_nginx_playbook.yml
- For prod inventory
ansible-playbook -i inventories/prod playbooks/install_nginx_playbook.yml
This will execute the playbook using the updated all(dev,stg,prod) inventory.
Step 7: Varify the all servers whether html page is visible or not (for all inventory like : dev,stg,prod):
9. Final Directory structure for this project
.
├── README.md
├── ansible
│ ├── inventories
│ │ ├── dev
│ │ ├── prod
│ │ └── stg
│ ├── playbooks
│ │ ├── install_nginx_playbook.yml
│ │ └── nginx-role
│ │ ├── README.md
│ │ ├── defaults
│ │ │ └── main.yml
│ │ ├── files
│ │ │ └── index.html
│ │ ├── handlers
│ │ │ └── main.yml
│ │ ├── meta
│ │ │ └── main.yml
│ │ ├── tasks
│ │ │ └── main.yml
│ │ ├── templates
│ │ ├── tests
│ │ │ ├── inventory
│ │ │ └── test.yml
│ │ └── vars
│ │ └── main.yml
│ └── update_inventories.sh
└── terraform
├── modules
│ ├── bucket.tf
│ ├── dynamodb.tf
│ ├── ec2.tf
│ ├── output.tf
│ └── variable.tf
├── main.tf
├── outputs.tf.tf
├── providers.tf
├── terraform.tf
├── terraform.tfstate
└── terraform.tfstate.backup
10. Infrastructure Destroy
To destroy the infrastructure, follow these simple steps:
Navigate to the Terraform Directory: Go to the directory where your Terraform configuration files are located. This is typically where your
main.tffile and other Terraform scripts are present.cd /path/to/terraform/directoryRun Terraform Destroy: Execute the following command to destroy all the resources that were created by Terraform. The
--auto-approveflag ensures that you won’t be prompted to confirm the destruction.terraform destroy --auto-approveThis command will:
Destroy all EC2 instances
Delete all S3 buckets
Remove any databases or other resources provisioned during the setup
Once the command finishes executing, your infrastructure will be completely torn down, and you will have successfully cleaned up all resources.