ClearLinux has caught my attention for its incredibly lean and fast boot/build/compile times, and its unbeatable speed/performance on Intel-based x86 EC2s.
For that reason, I’ve spent the last few months experimenting with Clear Linux on EC2 in my free time.
After spending 2 months tinkering with Clear Linux on AWS, I am having a few problems I can’t seem to figure out.
Problems
Problem #1: Packer + ClearLinux
It seems as though you cannot “bake” a custom Clear Linux AMI, and then use THAT custom baked AMI as a base to build a “derived” image.
Let’s say that I custom-build an AMI from the ClearLinux 37000 on the AWS Marketplace, and I name that new AMI “Image One.”
Now, let’s say that I want to use “Image One” as a base to build another custom image, named “Image Two.”
When I build Image One, and create an EC2 instance from that AMI → everything is fine.
All my packages and changes are there. But when I use Image One as the base Packer base for Image Two, NONE of my packages or configurations from Image One are present on the Packer builder’s Launched EC2 instance. It’s almost as if the AMI has been somehow “reset” or “wiped.”
Simple Example (full example is included further down below…)
- Bake a custom Clear Linux AMI from the latest Clear Linux AMI on AWS…
- Install
rpminto the custom image withswupd bundle-add package-utilsduring Packer build - Packer build completes → get back new AMI-ID → create EC2 from AMI
- Access the EC2 and check for rpm →
rpm=> command exists - Create a new Packer builder, and use the previous ^^ AMI as its base
- Packer Shell provisioner reports
rpm: command not foundwhen attempting to install a custom .rpm file generated withautospec - Access the Packer EC2 (using
-debugmode in Packer) --> check if the bundle and/or binary for rpm are present...but they are not. Therpm` binary is nowhere to be found…in fact, none of the packages I installed or changes I made to my custom AMI “base” image are present.
I’ve tried everything in order to get this to work, and I have no idea why I can’t build a “second-phase” or “derived” AMI from a customized “base” Clear Linux AMI.
I suspect this has something to do with Clear Linux’s “Stateless” principles, or maybe ucd is doing something I don’t understand…but I sincerely would appreciate help with this one.
PLEASE HELP 
Packer Stuff (Detailed)
-
Derive a “base” image (aka “Golden Image”) from the ClearLinux AMI →
- This image is basically just my systems baseline.
- Install a bunch ofswupd bundles
- Disableswupdautoupdates
- Add some default/etc/profileand/etc/environmentitems (NOT SURE IF THESE ARE IN THE RIGHT PLACE)
- Stop/mask tallow (ALL systems in our infrastructure are managed and remotely accessed only through theaws-ssm-agent, so we don’t need tallow)
{
"variables": {
...
"source_ami_arch": "x86_64",
"source_ami_name": "clear-3*",
"source_ami_owner": "679593333241",
"product-code": "4ks0lssia3phyyrbckdk9xgba",
"source_ami_ssh_user": "clear"
...
}
}
{
"type": "shell",
"expect_disconnect": true,
"inline": [
"SWUPD=$(sudo pidof swupd); while ps -p ${SWUPD} > /dev/null; do echo 'Waiting for swupd...'; sleep 1; done;",
"sudo swupd autoupdate --disable --no-progress",
"sudo systemctl stop tallow",
"sudo systemctl mask tallow",
"sudo systemctl daemon-reload",
"sudo mkdir -p ~/usr/bin",
"sudo mkdir -p ~/usr/lib64",
"sudo touch /etc/environment",
"sudo touch /etc/profile",
"echo 'PATH=$PATH:~/usr/bin' | sudo tee --append /etc/environment",
"echo 'PATH=$PATH:~/usr/bin' | sudo tee --append /etc/profile",
"echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/usr/lib64' | sudo tee --append /etc/environment",
"echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/usr/lib64' | sudo tee --append /etc/profile",
"sudo swupd bundle-add c-basic c-basic-legacy cloud-api cloud-control cloud-native-basic cpio cronie curl devpkg-expat devpkg-libnetfilter_conntrack devpkg-LVM2 devpkg-openssl devpkg-systemd flatpak go-basic-dev iperf ipvsadm jq kernel-aws-dkms logrotate network-basic nfs-utils openssl os-cloudguest-aws os-core os-core-search package-utils parted python-extras runtime-libs-boost rust-basic socat service-os-dev sysadmin-basic time-server-basic unzip wget yq",
"sudo shred -u /etc/ssh/*_key /etc/ssh/*_key.pub",
"shred -u ~/.ssh/authorized_keys",
"sudo shred -u /var/lib/cloud/aws-user-data",
"sudo shred -u /etc/machine-id"
]
}
^^ The above produces an AMI with the expected packages present (rpm, dnf, ifconfig...)
But when this image (the resulting AMI) is used later in a different Packer builder as the source_ami: for some other build, the Packer Shell provisioner reports rpm: command not found.
ANY help or guidance would be much appreciated. Thank you again for all of your work!