In this guide we will learn how to provision an EC2 instance with user_data when launching the instance using terraform.

AWS user_data is the set of commands/data you can provide to an instance at launch time. For example if you are launching an ec2 instance and want to have docker installed on the newly launched ec2, than you can provide set of bash commands in the user_data field of aws ec2 config page.

We can do this level of customization during the image build time with packer as well.

With terraform we can do it on the go and have different set of user_data for different set of machines you are launching with help of Loop or conditional statements.

Note: This guide assumes that you have terraform installed locally and access to an AWS account. I am using terraform version 1.1 in this guide.

Related content:

• How to use terraform targets to run specific resource
• Terraform AWS VPC with Public and Private subnets with NAT
• Using Terraform to Launch a VPS Instance in Digital Ocean
• Create an RDS instance in terraform with a Mariadb Example
• Using terraform to launch Digitaocean kubernetes cluster
• How to Create Aws Lightsail Instance With Terraform

Terraform code to create an ec2 with user_data

This is the terraform code to launch an ec2 with userdata to provision the server. We are also creating a security group to allow ssh and web access from the server. Update the script with your details. Change the following: aws region, ami ID, instance_type, count ( no of instances to launch), security group id, user_data ( replace it with whatever commands or script you want to execute), subnet_id.

provider "aws" {
  region = "eu-west-3"
}

module "ec2-instance" {
  source  = "terraform-aws-modules/ec2-instance/aws"
  version = "~> 3.0"
  count   = 1

  name                        = "ubuntusrv"
  ami                         = "ami-0cdd5983782a9dbe1"
  associate_public_ip_address = "true"
  disable_api_termination     = false
  instance_type               = "t3.medium"
  key_name                    = "etowett-key"
  monitoring                  = true
  subnet_id                   = "subnet-01c7d9a124dd4f05f"
  user_data                   = <<EOF
#!/bin/bash -xe
sudo apt update
sudo apt upgrade -y
sudo hostnamectl set-hostname ubuntusrv.citizix.com
sudo apt install -y nginx vim
sudo cat > /var/www/html/hello.html <<EOD
Hello world!
EOD
EOF

  vpc_security_group_ids = [
    aws_security_group.ec2-instance-sg.id
  ]

  root_block_device = [
    {
      volume_size           = 50
      volume_type           = "gp2"
      delete_on_termination = true
    },
  ]

  tags = {
    Name = "ubuntusrv"
  }
}

resource "aws_security_group" "ec2-instance-sg" {
  name        = "Allow web traffic"
  description = "Allow Web inbound traffic"
  vpc_id      = "vpc-09ea5a4576ba3aaba"

  ingress {
    description      = "Allow ssh from everywhere"
    from_port        = 22
    to_port          = 22
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  ingress {
    description      = "Allow http from everywhere"
    from_port        = 80
    to_port          = 80
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  ingress {
    description      = "Allow TLS from everywhere"
    from_port        = 443
    to_port          = 443
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  tags = {
    Name = "allow_web_traffic"
  }
}

output "server-private-ip" {
  value       = module.ec2-instance.*.private_ip
  description = "Instance PrivateIP"
}

output "server-public-ip" {
  value       = module.ec2-instance.*.public_ip
  description = "Instance PublicIP"
}

As part of user_data we are doing the following tasks:

• Updating the server
• Setting up the server hostname
• Installing vim and nginx
• Setting up a simple nginx hello world page

Since the count is set to 1 here we would be creating only one instance.

Terraform initializing and planing changes

Since we are using a terraform module terraform-aws-modules/ec2-instance/aws to launch the instance, we will need to initialize to download the module. Use this command to achieve that:

terraform init

Next let us do  Terraform Plan with the -out file. I recommend writting the plan as an out file and use it to make sure what you see is what you get.

terraform plan -out tf.plan

Here is my output on my server:

➜ terraform plan -out tf.plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
  + create

Terraform will perform the following actions:

  # aws_security_group.ec2-instance-sg will be created
  + resource "aws_security_group" "ec2-instance-sg" {
      + arn                    = (known after apply)
      + description            = "Allow Web inbound traffic"
      + egress                 = [
          + {
              + cidr_blocks      = [
                  + "0.0.0.0/0",
                ]
              + description      = ""
              + from_port        = 0
              + ipv6_cidr_blocks = [
                  + "::/0",
                ]
              + prefix_list_ids  = []
              + protocol         = "-1"
              + security_groups  = []
              + self             = false
              + to_port          = 0
            },
        ]
      + id                     = (known after apply)
      + ingress                = [
          + {
              + cidr_blocks      = [
                  + "0.0.0.0/0",
                ]
              + description      = "Allow TLS from everywhere"
              + from_port        = 443
              + ipv6_cidr_blocks = [
                  + "::/0",
                ]
              + prefix_list_ids  = []
              + protocol         = "tcp"
              + security_groups  = []
              + self             = false
              + to_port          = 443
            },
          + {
              + cidr_blocks      = [
                  + "0.0.0.0/0",
                ]
              + description      = "Allow http from everywhere"
              + from_port        = 80
              + ipv6_cidr_blocks = [
                  + "::/0",
                ]
              + prefix_list_ids  = []
              + protocol         = "tcp"
              + security_groups  = []
              + self             = false
              + to_port          = 80
            },
          + {
              + cidr_blocks      = [
                  + "0.0.0.0/0",
                ]
              + description      = "Allow ssh from everywhere"
              + from_port        = 22
              + ipv6_cidr_blocks = [
                  + "::/0",
                ]
              + prefix_list_ids  = []
              + protocol         = "tcp"
              + security_groups  = []
              + self             = false
              + to_port          = 22
            },
        ]
      + name                   = "Allow web traffic"
      + name_prefix            = (known after apply)
      + owner_id               = (known after apply)
      + revoke_rules_on_delete = false
      + tags                   = {
          + "Name" = "allow_web_traffic"
        }
      + tags_all               = {
          + "Name" = "allow_web_traffic"
        }
      + vpc_id                 = "vpc-09ea5a4576ba3aaba"
    }

  # module.ec2-instance[0].aws_instance.this[0] will be created
  + resource "aws_instance" "this" {
      + ami                                  = "ami-0cdd5983782a9dbe1"
      + arn                                  = (known after apply)
      + associate_public_ip_address          = true
      + availability_zone                    = (known after apply)
      + cpu_core_count                       = (known after apply)
      + cpu_threads_per_core                 = (known after apply)
      + disable_api_termination              = false
      + ebs_optimized                        = (known after apply)
      + get_password_data                    = false
      + host_id                              = (known after apply)
      + id                                   = (known after apply)
      + instance_initiated_shutdown_behavior = (known after apply)
      + instance_state                       = (known after apply)
      + instance_type                        = "t3.medium"
      + ipv6_address_count                   = (known after apply)
      + ipv6_addresses                       = (known after apply)
      + key_name                             = "etowett-key"
      + monitoring                           = true
      + outpost_arn                          = (known after apply)
      + password_data                        = (known after apply)
      + placement_group                      = (known after apply)
      + placement_partition_number           = (known after apply)
      + primary_network_interface_id         = (known after apply)
      + private_dns                          = (known after apply)
      + private_ip                           = (known after apply)
      + public_dns                           = (known after apply)
      + public_ip                            = (known after apply)
      + secondary_private_ips                = (known after apply)
      + security_groups                      = (known after apply)
      + source_dest_check                    = true
      + subnet_id                            = "subnet-01c7d9a124dd4f05f"
      + tags                                 = {
          + "Name" = "ubuntusrv"
        }
      + tags_all                             = {
          + "Name" = "ubuntusrv"
        }
      + tenancy                              = (known after apply)
      + user_data                            = "e45f58da4799de70b049740df48fb535335ad8d4"
      + user_data_base64                     = (known after apply)
      + volume_tags                          = {
          + "Name" = "ubuntusrv"
        }
      + vpc_security_group_ids               = (known after apply)

      + capacity_reservation_specification {
          + capacity_reservation_preference = (known after apply)

          + capacity_reservation_target {
              + capacity_reservation_id = (known after apply)
            }
        }

      + credit_specification {}

      + ebs_block_device {
          + delete_on_termination = (known after apply)
          + device_name           = (known after apply)
          + encrypted             = (known after apply)
          + iops                  = (known after apply)
          + kms_key_id            = (known after apply)
          + snapshot_id           = (known after apply)
          + tags                  = (known after apply)
          + throughput            = (known after apply)
          + volume_id             = (known after apply)
          + volume_size           = (known after apply)
          + volume_type           = (known after apply)
        }

      + enclave_options {
          + enabled = (known after apply)
        }

      + ephemeral_block_device {
          + device_name  = (known after apply)
          + no_device    = (known after apply)
          + virtual_name = (known after apply)
        }

      + metadata_options {
          + http_endpoint               = "enabled"
          + http_put_response_hop_limit = 1
          + http_tokens                 = "optional"
        }

      + network_interface {
          + delete_on_termination = (known after apply)
          + device_index          = (known after apply)
          + network_interface_id  = (known after apply)
        }

      + root_block_device {
          + delete_on_termination = true
          + device_name           = (known after apply)
          + encrypted             = (known after apply)
          + iops                  = (known after apply)
          + kms_key_id            = (known after apply)
          + throughput            = (known after apply)
          + volume_id             = (known after apply)
          + volume_size           = 50
          + volume_type           = "gp2"
        }

      + timeouts {}
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + server-private-ip = [
      + (known after apply),
    ]
  + server-public-ip  = [
      + (known after apply),
    ]

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: tf.plan

To perform exactly these actions, run the following command to apply:
    terraform apply "tf.plan"

Terraform applying changes

Now we apply the changes with the plan file. Ensure you are using the right out file otherwise you might end up destroying resources:

terraform apply tf.plan

You should see some output similar to this:

➜ terraform apply tf.plan
aws_security_group.ec2-instance-sg: Creating...
aws_security_group.ec2-instance-sg: Still creating... [10s elapsed]
aws_security_group.ec2-instance-sg: Still creating... [20s elapsed]
aws_security_group.ec2-instance-sg: Still creating... [30s elapsed]
aws_security_group.ec2-instance-sg: Still creating... [40s elapsed]
aws_security_group.ec2-instance-sg: Still creating... [50s elapsed]
aws_security_group.ec2-instance-sg: Still creating... [1m0s elapsed]
aws_security_group.ec2-instance-sg: Creation complete after 1m6s [id=sg-040bac55fc95ccf5f]
module.ec2-instance[0].aws_instance.this[0]: Creating...
module.ec2-instance[0].aws_instance.this[0]: Still creating... [10s elapsed]
module.ec2-instance[0].aws_instance.this[0]: Still creating... [20s elapsed]
module.ec2-instance[0].aws_instance.this[0]: Still creating... [30s elapsed]
module.ec2-instance[0].aws_instance.this[0]: Still creating... [40s elapsed]
module.ec2-instance[0].aws_instance.this[0]: Still creating... [50s elapsed]
module.ec2-instance[0].aws_instance.this[0]: Still creating... [1m0s elapsed]
module.ec2-instance[0].aws_instance.this[0]: Still creating... [1m10s elapsed]
module.ec2-instance[0].aws_instance.this[0]: Still creating... [1m20s elapsed]
module.ec2-instance[0].aws_instance.this[0]: Still creating... [1m30s elapsed]
module.ec2-instance[0].aws_instance.this[0]: Still creating... [1m40s elapsed]
module.ec2-instance[0].aws_instance.this[0]: Still creating... [1m50s elapsed]
module.ec2-instance[0].aws_instance.this[0]: Still creating... [2m0s elapsed]
module.ec2-instance[0].aws_instance.this[0]: Still creating... [2m10s elapsed]
module.ec2-instance[0].aws_instance.this[0]: Still creating... [2m20s elapsed]
module.ec2-instance[0].aws_instance.this[0]: Still creating... [2m30s elapsed]
module.ec2-instance[0].aws_instance.this[0]: Creation complete after 2m33s [id=i-08dea00d4a2cbe231]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Outputs:

server-private-ip = [
  "10.70.5.194",
]
server-public-ip = [
  "35.180.250.167",
]

Validating changes

Once the execution finishes, the changes should be applied. Hostname should be changed to what we set, confirm by logging in.

ubuntu@ubuntusrv:~$

Let us confirm that nginx is installed

$ apt-cache policy nginx
nginx:
  Installed: 1.18.0-0ubuntu1.2
  Candidate: 1.18.0-0ubuntu1.2
  Version table:
 *** 1.18.0-0ubuntu1.2 500
        500 http://eu-west-3.ec2.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages
        100 /var/lib/dpkg/status
     1.17.10-0ubuntu1 500
        500 http://eu-west-3.ec2.archive.ubuntu.com/ubuntu focal/main amd64 Packages

Finally let us confirm that the file for hello world was created

$ file /var/www/html/hello.html
/var/www/html/hello.html: ASCII text

Awesome. Now that everything is set up as expected, we can use this curl command to confirm that nginx is serving our page

$ curl http://35.180.250.167/hello.html
Hello world!

Using Shell script in Terraform user_data

In the previous example, we learnt how to use inline shell commands for our user_data. It is advisable to use shell script to keep things cleaner instead of commands and EOF.

The same Terraform script we have written earlier can be written like this and all the Shell commands we had between EOF be saved into a file, in our case init.sh. All those lines are now being replaced with a single line.

user_data = file("init.sh")

While referring the shell script with in file  the path is relative.  Simple init.sh means that the tf and the sh files are present on the same directory level.

Troubleshooting user_data related issues in AWS

Here are some pointers to remember

• User data shell scripts must start with the Shebang #! characters and the path to the interpreter you want to read the script (commonly /bin/bash).
• Scripts entered as user data are run as the root user, so no need to use the sudo command in the init script.
• When a user data script is processed, it is copied to and run from /var/lib/cloud/instances/instance-id/. The script is not deleted after it is run and can be found in this directory with the name user-data.txt So to check if your shell script made to the server refer this directory and the file.
• The cloud-init output log file (/var/log/cloud-init-output.log) captures console output of your user_data shell script. to know how your user_data shell script was executed and its output check this file.