Monday Night Live With Peter DeSantis

here we go! Monday Night Live with Peter DeSantis is about to kick off at @awscloud #reinvent 2022!

a 🧵👇 (/cc @AWSEvents)

@marknca tweeted at 29-Nov-2022, 03:31

@awscloud @AWSEvents I love how much fun Peter has with this keynote!

#reinvent

@marknca tweeted at 29-Nov-2022, 03:33

Peter reminds everyone that this keynote is all about "how" @awscloud does things. lots of behind the scenes info in this one..

#reinvent

@marknca tweeted at 29-Nov-2022, 03:34

👆 Start

big focus on performance tonight.

Peter reminds us it’s not all about what you can see

“Great performance is the result of innovation from the ground up and investing over time”

#reinvent

@marknca tweeted at 29-Nov-2022, 03:35

starting with a dive into @awscloud Nitro...

more on Nitro at https://aws.amazon.com/ec2/nitro/

#reinvent

@marknca tweeted at 29-Nov-2022, 03:36

here’s a great talk from Abby Fuller on the technical details of AWS Nitro: https://www.youtube.com/watch?v=e1_UJTgyFqM&feature=youtu.be

#reinvent

@marknca tweeted at 29-Nov-2022, 03:37

Peter announces a new version of Nitro (v5), with big gains behind the scenes. continued innovation behind the scenes from @awscloud here

#reinvent

@marknca tweeted at 29-Nov-2022, 03:39

NEW INSTANCE TYPE:

C7gn

uses the new Nitro v5 chip under the hood. lots of benefits. tip of the iceberg here

#reinvent

@marknca tweeted at 29-Nov-2022, 03:40

NEW CPU:

Graviton3E

custom tuned Graviton3 silicon. lots of benefits for the targeted use cases

#reinvent

@marknca tweeted at 29-Nov-2022, 03:41

NEW INSTANCE TYPE (coming soon):

HPC7g

…combines the Graviton3E + Nitro v5 for some big boosts for HPC applications

#reinvent

@marknca tweeted at 29-Nov-2022, 03:42

not sure I’m digging the graphic style for these slides...

#reinvent

@marknca tweeted at 29-Nov-2022, 03:42

typography with Amazon Ember is on point though. nice and clean. super clear. great typeface

#reinvent

@marknca tweeted at 29-Nov-2022, 03:43

back to the tech, here’s a great summary of the new instance types just announce from @jeffbarr: https://aws.amazon.com/blogs/aws/new-amazon-ec2-instance-types-in-the-works-c7gn-r7iz-and-hpc7g/

#reinvent

@marknca tweeted at 29-Nov-2022, 03:44

btw, more on the Graviton processor at https://aws.amazon.com/ec2/graviton/

#reinvent

@marknca tweeted at 29-Nov-2022, 03:45

Peter has moved on to how the @awscloud Global Infrastructure is designed and built

specifically talking about SRD (scalable reliable datagram)

more on that at: https://aws.amazon.com/blogs/hpc/in-the-search-for-performance-theres-more-than-one-way-to-build-a-network/

#reinvent

@marknca tweeted at 29-Nov-2022, 03:46

more on the @awscloud global infrastructure at https://aws.amazon.com/about-aws/global-infrastructure/

#reinvent

@marknca tweeted at 29-Nov-2022, 03:46

Peter is giving a great high level overview of TCP as a protocol

tl:dr it works great, but focuses on “getting there” vs. performance

at @awscloud scale, they need something more efficient and focused on hyper scale issues

#reinvent

@marknca tweeted at 29-Nov-2022, 03:48

key benefits of SRD on the @awscloud network:

• multi-pathing
• retries in microseconds
• runs on nitro (dedicated hardware)

#reinvent

@marknca tweeted at 29-Nov-2022, 03:49

remember, @awscloud runs at a completely different scale. very few companies run at this size. it’s a completely different set of problems, requiring completely different solutions

#reinvent

@marknca tweeted at 29-Nov-2022, 03:49

SRD is helping deliver better performance for Amazon EBS, especially for “tail latency” …average latency doesn’t matter if you just need the data!

#reinvent

@marknca tweeted at 29-Nov-2022, 03:51

teal line is SRD performance. big improvement overall, but massive in the edge cases

…btw, all new EBS io2 volumes will get this performance boost in the new year (stay tuned, they are just saying “coming soon")

#reinvent

@marknca tweeted at 29-Nov-2022, 03:54

NEW LAUNCH:

ENA Express, brings SRD to ENA…available today!

more details on ENA (basic, not express): https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networking-ena.html

#reinvent

@marknca tweeted at 29-Nov-2022, 03:55

here’s the SRD boost for ENA with ENA Express (needs to be enabled on your ENAs)

#reinvent

@marknca tweeted at 29-Nov-2022, 03:55

lots of improvements backed by the move to SRD in the @awscloud network

#reinvent

@marknca tweeted at 29-Nov-2022, 03:56

“A veritable hoedown of machine learning”, Peter DeSantis

🤦🤦🤦

@marknca tweeted at 29-Nov-2022, 03:57

oh Peter….

#reinvent

@marknca tweeted at 29-Nov-2022, 03:57

this graphic redeems part of the hoedown comment…

would have been better if he drove it out on stage 😉

#reinvent

@marknca tweeted at 29-Nov-2022, 03:58

gist of this section: 🧠 machine learning models are getting MASSIVE

they need more performance (processing, storage, network, etc.) in order to train properly. @awscloud innovations in performance engineering like SRD are helping drive these advances

#reinvent

@marknca tweeted at 29-Nov-2022, 03:59

here are the details on the @awscloud Trn1 instance type that Peter referenced: https://aws.amazon.com/ec2/instance-types/trn1/

#reinvent

@marknca tweeted at 29-Nov-2022, 04:01

while they make the point, graphs without units and numbers are tough for me to look at

that said, numbers and more specifics would actual make these slides weaker and muddle the point Peter is making

#reinvent

@marknca tweeted at 29-Nov-2022, 04:03

“Just as a refresher. Stochastic rounding is probabilistic rounding”, Peter

#reinvent

@marknca tweeted at 29-Nov-2022, 04:04

why is that important 👆? it helps train the more faster with less loss. @awscloud accomplishes this with hardware acceleration in the Trainium silicon

more on that chip at https://aws.amazon.com/machine-learning/trainium/

#reinvent

@marknca tweeted at 29-Nov-2022, 04:05

also, here’s a mathematical primer on stochastic rounding: https://nhigham.com/2020/07/07/what-is-stochastic-rounding/

you know, because MATH! 🤣

#reinvent

@marknca tweeted at 29-Nov-2022, 04:06

the tl:dr of this section is that @awscloud has been focusing on a lot of deeply interesting (to me at least!) performance engineering problems so that we can build & train ml 🧠 models faster and more accurately

#reinvent

@marknca tweeted at 29-Nov-2022, 04:06

how else would we get fun/weird avatar pics in different styles?

🤣🤣🤣

#reinvent

@marknca tweeted at 29-Nov-2022, 04:08

Peter is currently giving a great explanation of how to scale out a workload across multiple processors. I can’t explain it in 280 character chunks 🤦

basically: many hands make light work?

#reinvent

@marknca tweeted at 29-Nov-2022, 04:10

you’re going to have to watch this segment for yourself. it starts at 35m into this keynote with this slide 👇

#reinvent

@marknca tweeted at 29-Nov-2022, 04:11

Peter brings this back home using a Rings of Power reference. btw, the show is very much worth watching

https://t.co/kaILOWDPE6

#reinvent

@marknca tweeted at 29-Nov-2022, 04:12

lots of work to average out the results, but the “ring of rings” works much faster than “one ring”

…as long as you don’t give 9 to the race of man, 3 to the Elves, 7 for the Dwarves, and one to Sauron 🤦

#reinvent

@marknca tweeted at 29-Nov-2022, 04:15

new instance type coming soon to help speed things up even more:

Trn1n

• faster networking
• faster training

#reinvent

@marknca tweeted at 29-Nov-2022, 04:16

Jock Clear up now talking about the technical complexity of F1 racing

more details on @awscloud + F1 at https://aws.amazon.com/sports/f1/

#reinvent

@marknca tweeted at 29-Nov-2022, 04:17

three key aspects of making zoom-y cars zoom faster...

(somewhere @mosescj58 just felt a chill 🤣)

#reinvent

@marknca tweeted at 29-Nov-2022, 04:18

with @awscloud, Jock and the team built a virtual ground speed sensor

it’s an AI/ML tool that provides more accurate data faster…and more importantly with less weight

…would love to see more technical details on that one

#reinvent

@marknca tweeted at 29-Nov-2022, 04:19

btw, @QuinnyPig is also live tweeting this keynote. honestly, probably with a lot more—and funnier—jokes

check out his thread at https://twitter.com/QuinnyPig/status/1597430775330537472?s=20&t=reKJgcbkR9LjU2gLMZeRXg

#reinvent

@marknca tweeted at 29-Nov-2022, 04:22

Jock continues to lay out the challenges that F1 teams face outside of the race and the onslaught of challenges within a race

lots of engineering challenges and trade offs…

#reinvent

@marknca tweeted at 29-Nov-2022, 04:23

I’m not a car person (see the previous “zoom-y” comment), but I do love engineering challenges. lots of crazy ones here

Jock is constantly calling out the sheer number of decisions the team needs to make. they need data & analysis to make those decisions

#reinvent

@marknca tweeted at 29-Nov-2022, 04:25

they are launching an app with a bunch of race/interview content

…was hoping for an engineering data app like the real time MLB analysis: https://aws.amazon.com/blogs/machine-learning/calculating-new-stats-in-major-league-baseball-with-amazon-sagemaker/

..it’s not that, but I’m sure F1 fans will enjoy it

#reinvent

@marknca tweeted at 29-Nov-2022, 04:26

Peter is back up now. new topic: #serverless computing

#reinvent

@marknca tweeted at 29-Nov-2022, 04:27

“Behind me you can see a youthful @Werner announcing @awscloud Lambda”, Peter

😬🤣

#reinvent

@marknca tweeted at 29-Nov-2022, 04:27

Lambda has grown a LOT since it’s launch in 2014

thanks again @timallenwagner for setting us on this path!

#reinvent

@marknca tweeted at 29-Nov-2022, 04:28

Peter is diving into the technical details and some of the performance engineering that goes into @awscloud Lambda in order to reach this scale...

#reinvent

@marknca tweeted at 29-Nov-2022, 04:29

if you are interested in not only #serverless compute, but the whole concept. check out https://serverlessland.com/

#reinvent

@marknca tweeted at 29-Nov-2022, 04:30

#reinvent

@marknca tweeted at 29-Nov-2022, 04:30

Peter using a castle 🏰 analogy for virtual machines. key point: small number of access points. built for defence

#reinvent

@marknca tweeted at 29-Nov-2022, 04:31

“Mostly secure isn’t good enough”, Peter

#reinvent

@marknca tweeted at 29-Nov-2022, 04:31

#reinvent

@marknca tweeted at 29-Nov-2022, 04:31

“At @awscloud, we always use an isolated virtual machine for customer compute workloads”, Peter >> thus Nitro!

#reinvent

@marknca tweeted at 29-Nov-2022, 04:32

back to Lambda…

think of it as a cache. a big pool of compute

#reinvent

@marknca tweeted at 29-Nov-2022, 04:33

each slot was specific to a customer function. new function? new virtual machine

if a VM is up for the customer function, great. if not, initialize a new instance (empty & waiting) to handle it >> a/k/a cold starts

…it works, but there’s room for improvement

#reinvent

@marknca tweeted at 29-Nov-2022, 04:34

more graphs

#reinvent

@marknca tweeted at 29-Nov-2022, 04:35

so cold starts are a cache miss. that happens, caching isn’t perfect

#reinvent

@marknca tweeted at 29-Nov-2022, 04:36

obvious solution: bigger cache on the @awscloud backend

that’s more expensive and less efficient

enter Firecracker

#reinvent

@marknca tweeted at 29-Nov-2022, 04:37

more details on Firecracker: https://aws.amazon.com/blogs/aws/firecracker-lightweight-virtualization-for-serverless-computing/

#reinvent

@marknca tweeted at 29-Nov-2022, 04:37

moving to Firecrack provides more efficient use of the existing hardware. it also spins up faster, allowing things to sit in the cache longer!

#reinvent

@marknca tweeted at 29-Nov-2022, 04:38

results are Firecracker, bigger arrow, less bump

numbers would be helpful here. Peter mentioned one but I’m not sure I heard it right

assumption: things got better with the move to Firecracker

#reinvent

@marknca tweeted at 29-Nov-2022, 04:39

even with these improvements, you’ll see still cache misses (a/k/a cold starts)

need to make them faster. where is that time going?

for Java devs? a lot of it is in initialization

#reinvent

@marknca tweeted at 29-Nov-2022, 04:41

solution is to save the initialized start. think opening up your MacBook’s lid and working vs. turning it on from completely off

#reinvent

@marknca tweeted at 29-Nov-2022, 04:42

this feature is called AWS Lambda SnapStart and it’s available today!

bye bye cold starts…at least most of them

#reinvent

@marknca tweeted at 29-Nov-2022, 04:43

places where SnapStart _won’t_ work/help #1:

• some cryptography use cases where uniqueness is required
• can adjust with code changes

#reinvent #serverless

@marknca tweeted at 29-Nov-2022, 04:45

interesting division of snapshot-ing here. separating out the uniqueness to improve performance even further

#reinvent

@marknca tweeted at 29-Nov-2022, 04:46

last piece of the puzzle, predictive snapshot loading

get the pieces needed to start the function up and running as quickly as possible…before it’s even asked for

#reinvent

@marknca tweeted at 29-Nov-2022, 04:48

pumped to see the benefits of @awscloud Lambda SnapStat. love that I really don’t have to do anything to get those benefits

#reinvent #serverless

@marknca tweeted at 29-Nov-2022, 04:49

Peter signs off with some great session recommendations for performance engineering

#reinvent

@marknca tweeted at 29-Nov-2022, 04:50

great keynote from Peter DeSantis.hoping we’re going to see some @AmazonScience or @awscloud papers on these innovations soon

lots of great advancements & engineering work to deliver more features & functionality to builders

…more tomorrow as #reinvent kicks into high gear!

@marknca tweeted at 29-Nov-2022, 04:51

👆 Start