Private Yum Repository with Amazon S3

Mar 20, 2017 2 min read Dan aws linux python scripting

Setting up a public yum repository with S3 is straightforward, but making it private takes more effort. Access control has to be handled by a plugin that fetches files from the S3 bucket through the API, typically using stored credentials. On EC2 instances with an appropriate IAM role, you can skip storing credentials, but in any other environment that isn’t an option.

Either way, the first steps are the same for public and private setups.

1. Create the repository directory structure

Use the createrepo tool, which is available on both RedHat-based and Debian-based systems (e.g. Fedora/CentOS or Ubuntu). Running it creates a repodata subdirectory containing several files with metadata parsed from your RPM packages.

2. Sync the local repository with the S3 bucket

With the AWS Command Line Interface installed, this takes a single command:

    $ aws s3 sync ~/localrepo s3://yumrepobucket/remoterepo --delete

Running this from cron is trickier, because the credentials may not be picked up correctly. Setting the AWS_CONFIG_FILE environment variable should resolve that.

3. Install a yum plugin

You could write your own plugin if you know Python, but grabbing an existing one from GitHub is much quicker.

The one that worked best for me was yum-s3-iam. Installation was simple:

Copy s3iam.py to /usr/lib/yum-plugins/
Copy s3iam.conf to /etc/yum/pluginconf.d/
Copy a customized version of s3iam.repo to /etc/yum.repos.d/

Although the documentation doesn’t mention it, you can add the key_id and secret_key parameters to the .repo file. If you’re on EC2, configuring an IAM role instead should be enough. For security, whichever identity you use (role or user, if a role isn’t possible) should have read-only access to the bucket and nothing more.

That’s it: you now have a private yum repository hosted on an S3 bucket. Enjoy!