markn.ca
Archive · · 3 min read

Amazon S3 Glacier Re:Freezer

AWS Labs has a lot of open source code up on GitHub. This repo helps you visualize your AWS workloads and highlight areas of cost.

Amazon S3 Glacier Re:Freezer

AWS Labs has a lot of open source code up on GitHub. Amazon S3 Glacier Re:Freezer is one of those projects.

It’s got a simple premise: get the data out of Amazon S3 Glacier and back into Amazon S3.

That sounds simple but the solution is surprisingly nuanced. I call out a few more details in the Twitter thread below…

Tweet 1/10 👇 Next tweet

let's take a look at the Amazon S3 Glacier Re:Freezer project from @awscloud Labs the project is available at https://github.com/awslabs/amazon-s3-glacier-refreezer 🧵☁️ #cloud #devops

@marknca tweeted at 18-Nov-2021, 13:07

Tweet 2/10 👇 Next tweet 👆 Start

this thread can be see unrolled at https://t.co/4d3MidmVDs the previous thread about AWS Perspective is up at https://markn.ca/2021/aws-perspective/ 🧵☁️ #cloud #devops

@marknca tweeted at 18-Nov-2021, 13:07

Tweet 3/10 👇 Next tweet 👆 Start

the goal of this project is simple: get data out of @awscloud S3 Glacier and back into an Amazon S3 bucket that shouldn't be too hard... 🤣🤣🤣 🧵☁️ #cloud #devops

@marknca tweeted at 18-Nov-2021, 13:07

Tweet 4/10 👇 Next tweet 👆 Start

I laugh because this is the solution architecture 👇 🧵☁️ #cloud #devops

@marknca tweeted at 18-Nov-2021, 13:07

Tweet 5/10 👇 Next tweet 👆 Start

think about that for a minute. it builds out resources in SEVEN different @awscloud services just to move data from one service to another worse, those two services are intertwined already! if there was ever a solution crying out to be in product, it's this 🧵☁️ #cloud #devops

@marknca tweeted at 18-Nov-2021, 13:07

Tweet 6/10 👇 Next tweet 👆 Start

ok, rant aside (for now), this is a very useful solution. Glacier is the long term, cold storage (https://aws.amazon.com/s3/glacier/) where writing is easy and reading is slow and expensive Glacier is the "just in case" place for storage. 🧵☁️ #cloud #devop

@marknca tweeted at 18-Nov-2021, 13:07

Tweet 7/10 👇 Next tweet 👆 Start

Glacier is a great service because it's super low storage costs (https://aws.amazon.com/s3/pricing/) make it easy to keep things "available" you're charge to get data out and for those times you actually need that data, it's usually worth it 🧵☁️ #cloud #devop

@marknca tweeted at 18-Nov-2021, 13:07

Tweet 8/10 👇 Next tweet 👆 Start

you would think that restoration process is automation in a slick manner. it isn't natively this solution does a nice job of smoothing out this process simply: - point at the Glacier Vault - pick your retrieval tier - target bucket - target storage class 🧵☁️ #cloud #devop

@marknca tweeted at 18-Nov-2021, 13:07

Tweet 9/10 👇 Next tweet 👆 Start

the complexity in the architecture 👆 is because the data retrieval from @awscloud Glacier is glacially slow like a herd of 🐢🐢🐢 who aren't sure which way they are supposed to be going type of slow 🧵☁️ #cloud #devop

@marknca tweeted at 18-Nov-2021, 13:07

Tweet 10/10 👇 Next tweet 👆 Start

all of the different components in the design make sure that the data gets to the target bucket with a high level of reliability should this be part of the core service offering ? yes but it's not, so use this code /🧵☁️ #cloud #devop

@marknca tweeted at 18-Nov-2021, 13:07

👆 Start

Share Tweet Share Share Email

Read next