On SLAs and Uptime Guarantees

In this article, I look at the practical side of the “multiple nines” reliability figures that vendors love to advertise, and explain why a true 100% uptime is not something you can achieve.

SLA (Service Level Agreement) is a binding contract between a vendor and a customer. It is typically expressed as the percentage of a reference period (for example, a year) during which the service must work normally and deliver its intended output.

Uptime is the numeric part of that agreement, expressed either as a percentage or as a unit of time.

Note: These definitions don’t fully apply when a service is delivered from multiple infrastructure sets, such as different data centers serving different geographic regions. An outage in one region doesn’t mean every customer is affected, so a different calculation approach is needed. One option is to estimate the number of requests that went unserved during the outage, based on historical traffic data, and derive the availability figure from that.

To run the numbers yourself, you can use this calculator. As a rough guide:

• 99.9% allows about 9 hours of downtime per year.
• 99.95% allows just over 4 hours of downtime per year.
• 99.99% allows a little under 1 hour of downtime per year.
• 99.999% allows about 5 minutes of downtime per year.

Note: Amazon Web Services offers a 99.95% SLA for EC2 and EBS.

Downtime causes fall into two broad categories: planned and unplanned.

Planned downtime should be explicitly accounted for so that it fits within the SLA, possibly by using what’s known as an “error budget”. Unplanned downtime should also be anticipated and mitigated, though not every risk can be eliminated.

Planned outages, for example on a Linux web server, include:

• Operating system updates: If a kernel upgrade requires a reboot, a single restart can cost roughly 2 minutes. Three such restarts in a year are enough to rule out 99.999% uptime.
• Service updates: Depending on the service, downtime can be a few seconds, but it can also stretch to several minutes or even a significant fraction of an hour.

Unplanned outages range from hardware failures (such as a server dying) to software problems (such as a critical bug in the latest release). You should have mitigation plans for these scenarios to protect your SLA. Be aware, though, that in some situations the SLA simply can’t be preserved no matter how much effort you invest, and that possibility needs to be factored in too.

Note: Availability should also take into account the infrastructure between the end user and the service. Even if the service itself is perfectly available, a given user only experiences the uptime of their internet provider combined with that of every other component along the path. From that standpoint, offering an SLA above 99.9% may not bring much practical benefit to the average user, although it may still matter from a revenue-loss perspective.

When the unexpected happens, consider approaches like these:

• Rollback procedures: Have a way to revert to the last known good software version, such as restoring a snapshot or re-running deployment scripts pointed at the previous version. The DevOps team should be able to do this without developer assistance, and ideally it should be automated so anyone can trigger it at any hour.
• Failover architecture: Replicate the service across multiple nodes. This is the approach Amazon recommends for achieving an SLA above 99.95%.
• Cold standby or rapid redeployment: Keep a standby ready, or have procedures to quickly spin up a new node with the failed node’s configuration. This can be fully automated and triggered by a health check.

A 100% SLA is not achievable in any real-world scenario. That said, a failover architecture, such as multiple nodes serving the same service across different availability zones or regions, can go a long way toward earning those extra nines.

Note: This article was written by an AWS Certified Solutions Architect (Associate). Always work with an expert when setting up production environments.