Deploying HAProxy LoadBalancer on AWS Cloud with backend AWS Instances using Ansible

6 min readOct 26, 2020

In this guide we will be provisioning an HAProxy LoadBalancer on AWS Cloud and integrate it with multiple web servers running on AWS Instances.The whole setup will be deployed using Ansible.

To work with AWS instances we can use the Dynamic Inventory concept of Ansible in which as new instances are launched in the AWS cloud,their public IP will be retrieved dynamically and added to the Ansible inventory.

Create Dynamic Inventory for EC2

Dynamic inventory is basically a program or script written in any language which automatically adds hosts in the inventory depending on the conditions provided in the script.Here,we will be using a pre-created python script for EC2 as dynamic inventory.

Create a new folder /<any_name> and download the script using wget downloader.

[root@localhost myinventory]# wget https://raw.githubusercontent.com/ansible/ansible/stable-2.9/contrib/inventory/ec2.py
[root@localhost myinventory]# wget https://raw.githubusercontent.com/ansible/ansible/stable-2.9/contrib/inventory/ec2.ini

In the ec2.py file change the interpreter to python3 if you have pyhton3 installed in your system as follows

!#/usr/bin/python3

AWS has an SDK for python boto3 which should also be installed using pip installer

[root@localhost ~]# pip install boto3

For using AWS we have to provide our Access Key ID and Secret Access Key.We can provide AWS credentials using the following environment variables

[root@localhost ~]# export AWS_ACCESS_KEY_ID='Your access key id'
[root@localhost ~]# export AWS_SECRET_ACCESS_KEY='Your secret access key'

Configure Ansible on Controller Node

Ansible by default uses a file named ansible.cfg as it’s configuration file.Create a directory /etc/ansible and in this create a file ansible.cfg

In the configuration file we have to provide the path to our inventory,name of remote-user(optional) and the private key file which we will use as AWS key-pair for SSH connection.Additionally,we will be using ec2-user as sudo in AWS instance so we have to provide some privilege escalations as well.

[defaults]
inventory = /myinventory
host_key_checking= False
remote_user= ec2-user
private_key_file= mykey111.pem
[privilege_escalation]
become= true
become_user= root
become_method= sudo
become_ask_pass= false

Create roles for Launching AWS Instances for LoadBalancer and Webservers

In Ansible,we have to create a directory in which we will maintain all our roles and provide it’s path in ansible.cfg file as roles_path parameter.

For creating role,we can use ansible-galaxy command

[root@localhost myroles]# ansible-galaxy init aws_ec2
[root@localhost myroles]# ansible-galaxy init aws_webserver_instances

Here,we have created two roles,one for launching the ec2 instance which will work as LoadBalancer(aws_ec2) and the other for instances which will work as web servers(aws_webserver_instances).

This will create a folder with a number of sub-folders such as tasks,templates,vars etc in which we can maintain our different files.

For launching ec2 instance on AWS,we will use a module named ec2.In this,we will provide our VPC Subnet Id,AMI Id,Security group Id,tags,key name etc as properties.Since,this is a task we have to write this in the main.yml file of tasks folder.

# tasks file for aws_ec2- name: Lauching ec2 instance
  ec2:
    key_name: "{{ key }}"
    group_id: "{{ sg_id }}"
    instance_type: "{{ type }}"
    image: "{{ ami }}"
    region: "{{ region_name }}"
    wait: yes
    instance_tags:
      name: "{{ tag_name }}"
    count: 1
    vpc_subnet_id: "{{ subnet }}"
    assign_public_ip: yes# tasks file for aws_webserver_instances- name: Lauching ec2 instance
  ec2:
    key_name: "{{ key }}"
    group_id: "{{ sg_id }}"
    instance_type: "{{ type }}"
    image: "{{ ami }}"
    region: "{{ region_name }}"
    wait: yes
    count: 3
    instance_tags:
      name: "{{ tag_name }}"
    vpc_subnet_id: "{{ subnet }}"
    assign_public_ip: yes

Note that,in the task file of LoadBalancer the count is 1,whereas in the task file of Webservers the count is 3.

As you can see,we have mentioned the values of most properties inside braces “{{ some_value}}”.The values inside these braces are called variables.Now,that we have used these variables we have to provide values to these variables somewhere.The values of these variables are written in the main.yml file of vars folder.

# vars file for aws_ec2
key: 'mykey111'
sg_id: 'sg-041482d29f68b5ade'
type: 't2.micro'
ami: 'ami-0ebc1ac48dfd14136'
region_name: 'ap-south-1'
tag_name: 'LoadBalancer'
subnet: 'subnet-d9f5cfb1'# vars file for aws_webserver_instances
key: 'mykey111'
sg_id: 'sg-041482d29f68b5ade'
type: 't2.micro'
ami: 'ami-0ebc1ac48dfd14136'
region_name: 'ap-south-1'
tag_name: 'Webserver'
subnet: 'subnet-d9f5cfb1'

Now,when this role is executed in our final playbook it will create an AWS instance.This role will be followed by another role which will configure webserver and HAProxy LoadBalancer on these newly launched instances.But when ansible-playbook is executed the available hosts are listed initially before the instance is launched and at this point of time,there will be no hosts available.The host will get added dynamically to the inventory only when the instance gets launched successfully.Thus,we have to refresh our inventory before the role for launching webserver and loadbalancer gets executed.This can be done by using a module meta which has a property refresh_inventory.

Thus,add the following task in the tasks/main.yml file

- name: Refresh Inventory
  meta: refresh_inventory

Create Role for Webserver

For creating webserver role again use the ansible-galaxy init command.

In the configuration of webserver the steps required are as follows:

Install httpd software.This can be done using packages module.
Copy the contents of the webpage from the Controller node to the Managed Nodes/AWS Instances in the /var/www/html folder.This can be done using the copy module
Start httpd services using services module.

# tasks file for webserver_aws
- name: Install httpd
  package:
    name: "httpd"
    state: present
- name: Copy Contents of Webpage
  copy:
    content: "This is {{ ansible_hostname }}"
    dest: "/var/www/html/index.html"
- name: Start httpd
  service:
    name: "httpd"
    state: started

Create Role for LoadBalancer

Again use the ansible-galaxy init for creating a new role.

Now,in the Haproxy LoadBalancer setup we have to configure the LoadBalancer and provide the frontend port number and the IP addresses of the backend servers.All these configurations are done in the /etc/haproxy/haproxy.cfg file.

We will first install the haproxy software which comes along with the configuration file in the Controller Node.Make the necessary configurations and then transfer this file to the LoadBalancer.

Install haproxy in Controller Node

yum install haproxy

Make the necessary changes in the configuration file

In the configuration file we have to provide the frontend port and specify the webserver IPs as backend IPs.

We have to mention the frontend port number in the frontend main section in the bind property.

We have to mention the backend server IP addresses in the backend app section.In Ansible,the managed node IP addresses can be stored in groups.In the Dynamic Inventory for AWS we have the Instances grouped according to their tag names.Thus,all the instances with the tag name Webserver will be placed together in a group named “tag_name_Webserver”.

We can loop through this group “tag_name_Webserver” to add all the webservers present in it as backend servers.

Create final Playbook

Create a file <any_name>.yml and write the following plays in this file.

- hosts: localhost
  roles:
  -  aws_ec2
  -  aws_webserver_instances
- hosts: tag_name_Webserver
  roles:
  -  webserver_aws
- hosts: tag_name_LoadBalancer
  roles:
  -  loadbalancer

Thus,first the roles for launching AWS instances will be executed,then the role for configuring webservers will be executed and then finally the role for deploying loadbalancer will be executed.

Finally,run this playbook using ansible-playbook command.