Amazon S3 Glacier Re:Freezer

let's take a look at the Amazon S3 Glacier Re:Freezer project from @awscloud Labs

the project is available at https://github.com/awslabs/amazon-s3-glacier-refreezer

🧵☁️ #cloud #devops

@marknca tweeted at 18-Nov-2021, 13:07

this thread can be see unrolled at https://t.co/4d3MidmVDs

the previous thread about AWS Perspective is up at https://markn.ca/2021/aws-perspective/

🧵☁️ #cloud #devops

@marknca tweeted at 18-Nov-2021, 13:07

the goal of this project is simple:

get data out of @awscloud S3 Glacier and back into an Amazon S3 bucket

that shouldn’t be too hard…

🤣🤣🤣

🧵☁️ #cloud #devops

@marknca tweeted at 18-Nov-2021, 13:07

I laugh because this is the solution architecture 👇

🧵☁️ #cloud #devops

@marknca tweeted at 18-Nov-2021, 13:07

think about that for a minute. it builds out resources in SEVEN different @awscloud services just to move data from one service to another

worse, those two services are intertwined already!

if there was ever a solution crying out to be in product, it’s this

🧵☁️ #cloud #devops

@marknca tweeted at 18-Nov-2021, 13:07

ok, rant aside (for now), this is a very useful solution. Glacier is the long term, cold storage (https://aws.amazon.com/s3/glacier/) where writing is easy and reading is slow and expensive

Glacier is the “just in case” place for storage.

🧵☁️ #cloud #devop

@marknca tweeted at 18-Nov-2021, 13:07

Glacier is a great service because it's super low storage costs (https://aws.amazon.com/s3/pricing/) make it easy to keep things "available"

you’re charge to get data out and for those times you actually need that data, it’s usually worth it

🧵☁️ #cloud #devop

@marknca tweeted at 18-Nov-2021, 13:07

you would think that restoration process is automation in a slick manner. it isn't natively

this solution does a nice job of smoothing out this process

simply:

• point at the Glacier Vault
• pick your retrieval tier
• target bucket
• target storage class

🧵☁️ #cloud #devop

@marknca tweeted at 18-Nov-2021, 13:07

the complexity in the architecture 👆 is because the data retrieval from @awscloud Glacier is glacially slow

like a herd of 🐢🐢🐢 who aren’t sure which way they are supposed to be going type of slow

🧵☁️ #cloud #devop

@marknca tweeted at 18-Nov-2021, 13:07

all of the different components in the design make sure that the data gets to the target bucket with a high level of reliability

should this be part of the core service offering ? yes

but it’s not, so use this code

/🧵☁️ #cloud #devop

@marknca tweeted at 18-Nov-2021, 13:07

👆 Start