Automating safe, hands-off deployments

The Amazon Builder’s Library is a great set of deep dive papers into the challenges with modern systems. This post looks at how Amazon conducts hands-off deployments.

“Automating safe, hands-off deployments, takes a look at how Amazon does continuous deployment. Amazon has been doing this practice for a long while and has automated each process to the point where it’s hands-off.

I call out a few more details in the Twitter thread below…

Tweet 1/10 👇 Next tweet

in today's review, I'm looking at "Automating safe, hands-off deployments" by @clare_liguori at @awscloud

https://aws.amazon.com/builders-library/automating-safe-hands-off-deployments/

this one is all about getting your work out to your customers

🧵☁️ #cloud #devops

@marknca tweeted at 11-Nov-2021, 21:12

Tweet 2/10 👇 Next tweet 👆 Start

this thread is available unrolled at https://t.co/PIavI51500

yesterday's thread on "Fairness in multi-tenant systems" is up at https://markn.ca/2021/fairness-in-multi-tenant-systems/

🧵☁️ #cloud #devops

@marknca tweeted at 11-Nov-2021, 21:12

Tweet 3/10 👇 Next tweet 👆 Start

right out of the gate, I love @clare_liguori's callout, "continuous deployment pipelines to help us deploy quickly and safely"

two keys there: - quickly - safely

🧵☁️ #cloud #devops

@marknca tweeted at 11-Nov-2021, 21:12

Tweet 4/10 👇 Next tweet 👆 Start

another key quote from early on in the paper, "We identified deployment risks and found ways to mitigate those risks through new safety automation in pipelines" ❤️❤️❤️

spot on! modern deployment reduces risk when done well. keep that in mind #infosec folks

🧵☁️ #cloud #devops

@marknca tweeted at 11-Nov-2021, 21:12

Tweet 5/10 👇 Next tweet 👆 Start

the paper is roughly broken down in the parts of a pipeline:

- source - build - test - prod

...and of course how they all fit together

🧵☁️ #cloud #devops

@marknca tweeted at 11-Nov-2021, 21:12

Tweet 6/10 👇 Next tweet 👆 Start

one key point is that _everything_ in production at Amazon gets there via a pipeline. static assets, code, infrastructure, everything

these pipelines make it easier to keep track of the history of changes & to ensure a high bar for quality

🧵☁️ #cloud #devops

@marknca tweeted at 11-Nov-2021, 21:12

Tweet 7/10 👇 Next tweet 👆 Start

👆this also make it easier to do code reviews, as they are all going through the pipelines. a nice example of a code review checklist in presented in the 📑

🧵☁️ #cloud #devops

@marknca tweeted at 11-Nov-2021, 21:12

Tweet 8/10 👇 Next tweet 👆 Start

testing is discussed in depth & honestly there is a lot more testing than I expected. it's an example of how Amazon's approach has matured & takes advantage of the @awscloud environment

🧵☁️ #cloud #devops

@marknca tweeted at 11-Nov-2021, 21:12

Tweet 9/10 👇 Next tweet 👆 Start

deployments themselves are a bit crazy. that, I did expect.

the pipelines need to ensure that they don't negatively impact production...especially at scale.

@clare_liguori does a great job of breaking down the concerns & techniques used here

🧵☁️ #cloud #devops

@marknca tweeted at 11-Nov-2021, 21:12

Tweet 10/10 👇 Next tweet 👆 Start

📑 finishes off w/idea of pipelines as code. very meta but it makes sense given the number of pipelines in use

if you're deploying to production, you should check out this paper. that's a nice way of saying you should go read this paper ASAP 🤣

/🧵☁️ #cloud #devops

@marknca tweeted at 11-Nov-2021, 21:12

👆 Start