Introduction

The images you use to deploy in cloud environments such as AWS and Azure are pretty good. I mostly used Canonical’s Ubuntu and Amazon Linux but I have also started using RockyLinux quite recently.

As good as they are, they’re very basic. You will need to install all your software and configurations every time a new Linux virtual machine is created. There are many reasons you would want your image:

• Speed up deployments
• To apply security requirements (ie, PCI-DSS)
• To add monitoring such as prometheus node exporter
• To prepare images you would use auto-scaling groups preconfigured for the application set you’re scaling
• etc

For example, the project I’m working on right now for a financial services company requires that all virtual machines be deployed with a minimum of PCI-DSS.

I’m achieving this by using this great Ansible role by RedHat:

github.com

The issue is the playbook that uses this role can take up to 40 minutes in small instances. This is way too long for me, and I’m very impatient 🙄

The solution? To create a pre-configured image with the playbook configurations.

Packer

Packer is a well-known tool created by HashiCorp and the community to create custom images. It simply starts up a temporary virtual machine in any of the supported clouds (or on-premises using providers such as VMWare or VirtualBox) to use as a template.

Once the VM is started, it’ll run one or several of the provisioners you have set up to configure it. After this, the image can be saved to be used for your deployments.

AWS

AWS and Azure are not too dissimilar. I’m using the same Ansible playbook for both.

The first thing you need is to decide and locate the AMI image you’re going to use as a base. You can use the AWS CLI or the Azure CLI to list them. A couple of examples below:

aws ec2 describe-images --owners self amazon aws-marketplace \
  --filter "Name=name,Values=CentOS*"
aws ec2 describe-images --owners self amazon aws-marketplace \
  --filter "Name=name,Values=Rocky*"
aws ec2 describe-images --owners self aws-marketplace \
  --filter "Name=name,Values=CentOS_Stream_9*"

# Azure
az vm image list --offer rockylinux-x86_64 --all --output table

You will need from there the owner and the name to use in the packer configuration. The configuration below uses Rocky, but it shouldn’t take you much to change it to use CentOS or any other Linux flavour.

packer {
  required_plugins {
    amazon = {
      source  = "github.com/hashicorp/amazon"
      version = "~> 1"
    }
    ansible = {
      source  = "github.com/hashicorp/ansible"
      version = "~> 1"
    }
  }
}

locals {
  extra_args   = ["-v", "-e", "ansible_fqdn=base",
    "-e", "env=${var.env}",
    "-e", "arch=${var.arch}",
    "--scp-extra-args", "'-O'",
    "-e", "packer=true"]
  timestamp    = regex_replace(timestamp(), "[- TZ:]", "")
  architecture = var.arch == "amd64" ? "x86_64" : "aarch64"
}

variable "env" {
  type    = string
  default = "dev"
}

variable "arch" {
  type = string
}

variable "instance_type" {
  type    = string
  default = "t2.micro"
}

variable "availability_zone" {
  type    = string
  default = "eu-west-2b"
}

variable "region" {
  type    = string
  default = "eu-west-2"
}

source "amazon-ebs" "rocky" {
  ami_name                    = "base-${var.arch}-${local.timestamp}"
  instance_type               = var.instance_type
  region                      = var.region
  associate_public_ip_address = true
  source_ami_filter {
    filters = {
      name = "Rocky-9-EC2-Base-9.3-*.${local.architecture}-*"
    }
    most_recent = true
    owners      = ["792107900819", "679593333241"]
  }
  subnet_filter {
    filters = {
      "tag:Name" : "public-${var.availability_zone}"
    }
    most_free = true
    random    = false
  }
  ssh_username = "rocky"
}

build {
  name = "tkn-base-${var.arch}-${local.timestamp}"
  sources = [
    "source.amazon-ebs.rocky"
  ]

  provisioner "ansible" {
    command                 = "ansible-playbook"
    playbook_file           = "main.yml"
    user                    = "rocky"
    inventory_file_template = "controller ansible_host={{ .Host }} ansible_user={{ .User }} ansible_port={{ .Port }}\n"
    extra_arguments         = local.extra_args
    galaxy_file             = "requirements.yml"
  }
}

• Change the region as appropriate
• Check out the subnet_filter and adjust it to match a subnet in your VPC with internet access
• Create an Ansible playbook (I called mine main.yml with all the configurations you require in your VMs
• I’m using Ansible Galaxy. If you don’t need it, remove the galaxy_file config

Azure

Azure is quite similar. You have the same requirements as with AWS. See the example config below. Azure requires you to install the waagent. Make sure your Ansible playbook does it.

This is my Ansible snippet for it.

- name: Azure packages
      when:
        - azure is defined
        - azure | bool
      tags: azure
      block:
        - name: Install Azure agent
          ansible.builtin.package:
            name:
              - WALinuxAgent
              - cloud-init
              - cloud-utils-growpart
              - gdisk
              - hyperv-daemons

        - name: Enable Azure agent service
          ansible.builtin.systemd:
            enabled: true
            name: waagent
            state: started

And this is the packer config for Azure. I’m using again Rock Linux but you can change it to whichever flavour you prefer.

packer {
  required_plugins {
    azure = {
      source  = "github.com/hashicorp/azure"
      version = "~> 1"
    }
    ansible = {
      source  = "github.com/hashicorp/ansible"
      version = "~> 1"
    }
  }
}

locals {
  extra_args   = ["-vv",
    "-e", "ansible_fqdn=base",
    "-e", "env=${var.env}",
    "-e", "arch=${var.arch}",
    "--scp-extra-args", "'-O'",
    "-e", "packer=true", "-e", "azure=true"]
  timestamp    = regex_replace(timestamp(), "[- TZ:]", "")
  architecture = var.arch == "amd64" ? "x86_64" : "aarch64"
}

variable "env" {
  type    = string
  default = "dev"
}

variable "arch" {
  type = string
}

variable "instance_type" {
  type    = string
  default = "Standard_B1ms"
}

variable "region" {
  type    = string
  default = "uksouth"
}

source "azure-arm" "rocky" {
  azure_tags = {
    repo = "ap-common"
  }
  client_id                         = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx"
  client_secret                     = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx"
  image_offer                       = "rockylinux-${local.architecture}"
  image_publisher                   = "resf"
  image_sku                         = "9-base"
  location                          = var.region
  os_type                           = "Linux"
  subscription_id                   = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx"
  tenant_id                         = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx"
  vm_size                           = var.instance_type
  ssh_username                      = "packer"
  ssh_password                      = "PackerP4sswordNotSecure@"
  ssh_pty                           = true
  managed_image_resource_group_name = "DevOps"
  managed_image_name                = "base-${var.arch}-${local.timestamp}"
  plan_info {
    plan_name      = "9-base"
    plan_product   = "rockylinux-x86_64"
    plan_publisher = "resf"
  }
}

build {
  name = "base-${var.arch}-${local.timestamp}"
  sources = [
    "source.azure-arm.rocky"
  ]

  provisioner "ansible" {
    command                 = "ansible-playbook"
    playbook_file           = "main.yml"
    user                    = "packer"
    inventory_file_template = "controller ansible_host={{ .Host }} ansible_user={{ .User }} ansible_port={{ .Port }}\n"
    extra_arguments         = local.extra_args
    galaxy_file             = "requirements.yml"
  }

  provisioner "shell" {
    execute_command = "chmod +x {{ .Path }}; {{ .Vars }} sudo -E sh '{{ .Path }}'"
    inline          = ["/usr/sbin/waagent -force -deprovision+user &"]
    inline_shebang  = "/bin/sh -x"
  }
}

Running

It’s very simple. Install packer first (obviously) and initialise it with:

packer init my-config.pkr.hcl

Then you can run it with the config file as an argument and any runtime variables to the command line to customise the build:

# AWS
packer build my-config.pkr.hcl -var=env=dev \
  -var=instance_type=t3.small
packer build my-config.pkr.hcl -var=env=dev \
  -var=instance_type=t4g.medium -var=arch=arm64

# Azure
packer build my-config.pkr.hcl -var=env=dev \
  -var=instance_type=Standard_B1ms

Conclusion

It does take time to get it working. You’ll likely find your Ansible playbook fails until you get it right, and it’s difficult to diagnose. But it’s worth the effort as once you get your images ready you will be saving a lot of time in your automation and build process.

More importantly, if, like us, you’re security-minded, you’ll want to use a Linux VM that has been hardened with good standards.

As usual, if you need help, get in touch.

I, for one welcome our new robot overlords.

‍