By Jeremy Pedersen
Welcome back for week 2 of our Alibaba Cloud Q&A blog series.
This week, I'm going to address just one question, because it's a big one!
I'm paraphrasing a bit here, but I really did have a customer ask me if Alibaba Cloud was "nuclear proof".
Let's roll up our sleeves and tackle this one step-by-step.
Let's start by ruling out a global nuclear conflict between the major world powers, because that is a literal armageddon from which no living thing is safe. Friends, the world is ending: do you really care if your servers are still up?
Instead, let's use an imaginary terrorist attack as our scenario. Somehow, a terrorist group has built a nuclear weapon. One of two things could happen:
In the first scenario, we will assume a terrorist organization has successfully detonated a real, functional nuclear weapon. In the second, we will assume that the detonation was only partially successful (a fizzle) in nuclear bomb parlance) or that the weapon was a "dirty bomb", a conventional bomb intended to spread nuclear material over a wide area, forcing an evacuation and cleanup of the resulting fallout, but not necessarily a nuclear explosion.
As with any task in systems architecture or design, we need to decide what it means to succeed, and what metrics we should use to measure success.
Let's look at two metrics:
For each of these metrics, we will define 3 failure states:
Great! Now let's design a couple of different system architectures and see how each one responds to being nuked!
You are the lead systems architect for the Acme Widget Corporation (WidgeCo for short). You have been given the task of designing a web application. You are going to need one or more web servers (Alibaba Cloud ECS), a database (Alibaba Cloud RDS), object storage (Alibaba Cloud OSS), and load balancers (Alibaba Cloud SLB). You will also use Alibaba Cloud DNS to map your hostname to the public IP address of your load balancer.
Alibaba Cloud breaks its services down by Region and Zone. A Zone is a single Alibaba Cloud datacenter. All zones are physically separate from one another and use independent networking, power, and cooling equipment. A Region is a geographical area (like Kuala Lumpur or Beijing) that contains multiple Zones.
You are considering 3 potential architectures:
Your database, web server(s), object storage, and load balancer are all located in a single Alibaba Cloud Zone (datacenter). You are not using any of the Multi-Zone features built into SLB, RDS, or OSS.
Survivability: This architecture is vulnerable to a single-Zone failure: loss of cooling, power, or networking would cause downtime but should not cause data loss. A major disaster like a fire or explosion could cause either partial or total data loss, depending on your backup policy.
Your web servers are deployed across multiple Zones. You are using the built-in multi-Zone capabilities of Alibaba Cloud SLB and RDS. You have turned on ZRS (Zone Redundant Storage) for your OSS buckets, and you are making regular backups of your ECS Cloud Disks.
Survivability: This architecture will survive the complete loss of a single Zone with no data loss and no downtime (but potentially some performance degradation). This architecture is vulnerable to the loss of multiple Zones within a Region.
Your web servers are deployed across multiple Zones in multiple Alibaba Cloud Regions. You are using Alibaba Cloud DTS to synchronize data between RDS databases in different Regions. You are using DNS health checks and load balancing to distribute traffic between load balancers in multiple Alibaba Cloud Regions. You have turned on Cross Region Replication (CRR) for your OSS buckets, to automatically replicate data across Regions.
Survivability: This architecture can survive the total loss of all Zones within a single Region. This architecture is vulnerable to the loss of multiple Regions.
A terrorist organization sets off a dirty bomb or a partially successful nuclear weapon. Damage from the blast is limited, but radioactive fallout forces an evacuation. An Alibaba Cloud Zone is damaged in the blast, or is without power, networking, or cooling as a result of the attack.
Single-Zone Architecture:
- Data Loss: Partial (some data may be lost depending on backup settings)
- Downtime: Partial (temporary: system is recoverable)
Multi-Zone Architecture:
- Data Loss: None (most or all data survives)
- Downtime: None (no downtime)
Multi-Region Architecture:
- Data Loss: None (all data survives)
- Downtime: None (no downtime)
A terrorist organization succeeds in detonating a low-yield nuclear weapon. An Alibaba Cloud Zone is near ground zero. That Zone is completely destroyed. Other Zones may be damaged or lose power, networking, or cooling.
Single-Zone Architecture:
- Data Loss: Total (some data may be recoverable depending on backup policy)
- Downtime: Total (unrecoverable)
Multi-Zone Architecture:
- Data Loss: None (most or all data survives)
- Downtime: Partial (some downtime)
Multi-Region Architecture:
- Data Loss: None (all data survives)
- Downtime: None (no downtime)
A terrorist organization succeeds in detonating a large nuclear weapon, completely devastating a city. All Alibaba Cloud Zones within the affected Region are completely destroyed.
Single-Zone Architecture:
- Data Loss: Total (unrecoverable)
- Downtime: Total (unrecoverable)
Multi-Zone Architecture
- Data Loss: Total (unrecoverable)
- Downtime: Total (unrecoverable)
Multi-Region Architecture:
- Data Loss: None (most or all data survives)
- Downtime: None (no downtime)
Although a nuclear attack is not a likely scenario, we can learn a lot about disaster planning from this hypothetical exercise.
First, it's important to plan ahead for both large and small scale disasters by building failover and redundancy into your application from the start.
Whenever possible, avoid single points of failure (SPoFs) by backing data up across multiple zones, taking advantage of multi-zone availability features in Alibaba Cloud products, and even building across multiple regions as time and budget permits.
This will protect you against the small and mundane (power outages, fires, human error) and the large and unpredictable (hurricanes, earthquakes, tsunamis).
Great! Reach out to me at jierui.pjr@alibabacloud.com and I'll do my best to answer in a future Friday Q&A blog.
You can also follow the Alibaba Cloud Academy LinkedIn Page. We'll re-post the more interesting questions and answers there.
JDP - March 11, 2022
JDP - July 9, 2021
JDP - December 17, 2021
JDP - August 27, 2021
JDP - June 11, 2021
JDP - May 20, 2021
Application High Availability Service
Application High Available Service is a SaaS-based service that helps you improve the availability of your applications.
Learn More
Database Security Solutions
Protect, backup, and restore your data assets on the cloud with Alibaba Cloud database services.
Learn More
Elastic High Performance Computing Solution
High Performance Computing (HPC) and AI technology helps scientific research institutions to perform viral gene sequencing, conduct new drug research and development, and shorten the research and development cycle.
Learn MoreMore Posts by JDP
