Swami Sivasubramanian's Keynote at AWS re:Invent 2021

. @SwamiSivasubram up now for the #ml keynote at @awscloud #reinvent

🧵

@marknca tweeted at 01-Dec-2021, 16:32

Swami is setting the scene; data is everywhere. there’s mountains of it. that makes it hard to get value from it

#reinvent

@marknca tweeted at 01-Dec-2021, 16:34

“Survival of the most informed”, @SwamiSivasubram

#reinvent

@marknca tweeted at 01-Dec-2021, 16:35

quick little nod to swimming Australia. they are using @awscloud #ml services to pull insights from their training data

#reinvent

@marknca tweeted at 01-Dec-2021, 16:35

name drops for @NasdaqTech, @Philips, and @carrier … all previous customer references

#reinvent

@marknca tweeted at 01-Dec-2021, 16:37

now walking through an e-commerce application example

#reinvent

@marknca tweeted at 01-Dec-2021, 16:37

step 1: shove things into an RDBMS

#reinvent

@marknca tweeted at 01-Dec-2021, 16:37

step 2: realize that breaks at some point. time to use specific data stores that align to your data & it’s use

#reinvent

@marknca tweeted at 01-Dec-2021, 16:38

step 3: snag the streaming data too

#reinvent

@marknca tweeted at 01-Dec-2021, 16:38

step 4: start asking questions to draw out insights

#reinvent

@marknca tweeted at 01-Dec-2021, 16:39

step 5: realize a data lake is going to help unite all of these data sources and questions

#reinvent

@marknca tweeted at 01-Dec-2021, 16:39

step 6: move beyond analysis and visualization to prediction

#reinvent

@marknca tweeted at 01-Dec-2021, 16:40

…that obviously isn’t that clean a process and takes time but the gist is spot on

it’s a journey and it’s critical to use the right tool at the right time

#reinvent

@marknca tweeted at 01-Dec-2021, 16:41

ooohhh, nice shout out to the need for #security and #privacy controls throughout this process

#reinvent

@marknca tweeted at 01-Dec-2021, 16:41

good summary slide for the journey that @SwamiSivasubram’s example took us on

#reinvent

@marknca tweeted at 01-Dec-2021, 16:42

tl:dr on the sales/marketing side: @awscloud has a ton to offer in this area

#reinvent

@marknca tweeted at 01-Dec-2021, 16:42

lots of #ml adoption in the @awscloud community

#reinvent

@marknca tweeted at 01-Dec-2021, 16:43

btw, I’m betting on/hoping for at least @mza appearance in this keynote!

#reinvent

@marknca tweeted at 01-Dec-2021, 16:44

3 key elements to a modern end-to-end data strategy…

#reinvent

@marknca tweeted at 01-Dec-2021, 16:44

1. modernize => move away from on-premises & the 90’s? 🤣 2. unify => take the data lake approach 3. innovate => create new experiences & draw new insights

#reinvent

@marknca tweeted at 01-Dec-2021, 16:45

diving deeper into “modernize”, we have a BMW video segment

#reinvent

@marknca tweeted at 01-Dec-2021, 16:46

surprising no one, @awscloud is the platform for the BMW connected car initiative (currently 15 million connected vehicles)

#reinvent

@marknca tweeted at 01-Dec-2021, 16:46

. @SwamiSivasubram is cruising here. very quick moving keynote…

#reinvent

@marknca tweeted at 01-Dec-2021, 16:48

this slide should’ve said:

Managing infrastructure, that’s so 90’s 🤣

#reinvent

@marknca tweeted at 01-Dec-2021, 16:48

yup. 100%…

#reinvent

@marknca tweeted at 01-Dec-2021, 16:48

little update on/pitch for @awscloud Aurora << if you’re in the market for an RDBMS solution, start here

more on Aurora at https://t.co/8QVWyGjjjR

#reinvent

@marknca tweeted at 01-Dec-2021, 16:49

as usual @QuinnyPig is also covering the keynote. his thread is up at https://twitter.com/QuinnyPig/status/1466082818056810499

#reinvent

@marknca tweeted at 01-Dec-2021, 16:50

ohhh, new feature/service coming…you can just feel it

#reinvent

@marknca tweeted at 01-Dec-2021, 16:50

yup…definitely something coming around using #ml to predict data store ops issues

#reinvent

@marknca tweeted at 01-Dec-2021, 16:51

NEW FEATURE/SERVICE: Amazon DevOps Guru for RDS < it’s DevOps Guru pointed at RDS

GA today

#reinvent

@marknca tweeted at 01-Dec-2021, 16:52

more on the launch at https://t.co/waM99vGP27

#reinvent

@marknca tweeted at 01-Dec-2021, 16:52

I ❤️ these services because they leverage the scale & insight of @awscloud to help the rest of us build better

#reinvent

@marknca tweeted at 01-Dec-2021, 16:53

ohhh, are we about to get another “Get off proprietary databases” service?

#reinvent

@marknca tweeted at 01-Dec-2021, 16:53

Amazon RDS Customer flew under the radar for me

more on the service at https://t.co/EPCeIcH6rf

<< the announcement today was that it now supports SQL Server as well. GA today

#reinvent

@marknca tweeted at 01-Dec-2021, 16:54

…come on @awscloud Neptune #serverless…

please?

#reinvent

@marknca tweeted at 01-Dec-2021, 16:55

nope (or not yet?). I think we’re going in a different direction for the moment

#reinvent

@marknca tweeted at 01-Dec-2021, 16:55

EXCELLENT QUESTION …and one that not enough builders are asking (especially in legacy environments)

#reinvent

@marknca tweeted at 01-Dec-2021, 16:56

. @awscloud DynamoDB might be processing a few requests at any given time 🤣

#reinvent

@marknca tweeted at 01-Dec-2021, 16:57

ok, getting the setup for a new feature/service again

#reinvent

@marknca tweeted at 01-Dec-2021, 16:58

sidenote: this feels like more a @ajassy or @werner style of keynote. problem build => new feature/service

#reinvent

@marknca tweeted at 01-Dec-2021, 16:58

NEW FEATURE: Amazon DynamoDB Standard-Infrequent Access table class << this should help reduce some costs for people and allow them to keep data in DynamoDB longer

can’t wait to hear @alexbdebrie’s thoughts

#reinvent

@marknca tweeted at 01-Dec-2021, 16:59

more on the new table class at https://t.co/wytZzcvH9D

#reinvent

@marknca tweeted at 01-Dec-2021, 16:59

btw, if you’re working with @awscloud DynamoDB you NEED @alexbdebrie’s book: https://www.dynamodbbook.com/

it’s a life saver

#reinvent

@marknca tweeted at 01-Dec-2021, 17:00

also, watch one of his million+ talks on DynamoDB. only person I’ve seen to get a two-part talk at @awscloud #reinvent. that one happened last year

@marknca tweeted at 01-Dec-2021, 17:00

back to the keynote, lots of purpose-built databases from @awscloud

#reinvent

@marknca tweeted at 01-Dec-2021, 17:01

. @SwamiSivasubram touching on the challenges around migration of databases to @awscloud now

#reinvent

@marknca tweeted at 01-Dec-2021, 17:02

more on the @awscloud Data Migration Service (DMS) at https://t.co/GBPfOnoAI6

#reinvent

@marknca tweeted at 01-Dec-2021, 17:02

NEW FEATURE: @awscloud Database Migration Service Fleet Advisor << migration your databases en masse. it routes the data through S3 and customized migration plans to align w/the right purpose-built service

#reinvent

@marknca tweeted at 01-Dec-2021, 17:04

shifting to the “Unify” point now…

#reinvent

@marknca tweeted at 01-Dec-2021, 17:04

. @Expedia up as the customer example

#reinvent

@marknca tweeted at 01-Dec-2021, 17:05

nothing in the customer example other than they are using @awscloud 😕

#reinvent

@marknca tweeted at 01-Dec-2021, 17:05

#reinvent

@marknca tweeted at 01-Dec-2021, 17:05

key point here is that your data should be in one place and then pulled into the right tool at the right time >> data lake

#reinvent

@marknca tweeted at 01-Dec-2021, 17:06

new to the whole “data lake” thing? here’s a primer from @awscloud: https://t.co/JdyiUBiO6l

#reinvent

@marknca tweeted at 01-Dec-2021, 17:07

…and if you’re just getting started, use @awscloud Lake Formation and save yourself a ton of time

more on that at https://t.co/MqGU3Dl0xJ

#reinvent

@marknca tweeted at 01-Dec-2021, 17:07

all of these strategies start with @awscloud S3. it’s the easiest, least expensive, and most performant way to store the data at scale

#reinvent

@marknca tweeted at 01-Dec-2021, 17:08

# reinvent

@marknca tweeted at 01-Dec-2021, 17:08

if you’re not using Lake Formation, your “lake” is probably going to end up more like a pit. that’s bad

don’t do that

#reinvent

@marknca tweeted at 01-Dec-2021, 17:09

. @awscloud Athena is my 2nd favourite service (behind QuickSight) and often overlooked

more on Athena at https://t.co/mZ43E2kUxc

#reinvent

@marknca tweeted at 01-Dec-2021, 17:09

ohhh, are we getting some improvements to Athena?!?!

#reinvent

@marknca tweeted at 01-Dec-2021, 17:10

#reinvent

@marknca tweeted at 01-Dec-2021, 17:10

. @awscloud also takes the “purpose-built” approach when it comes to analytics

interactive => Athena big data => EMR ops & logs => OpenSearch real-time => Kinesis & MSK warehouse => Redshift

(roughly)

#reinvent

@marknca tweeted at 01-Dec-2021, 17:11

👆 most of those are now available #serverless. which is f—king awesome, btw

#reinvent

@marknca tweeted at 01-Dec-2021, 17:12

RedShift is under utilized by builders. mainly because it’s terrifying…specially when you look at cost

the #serverless announcement yesterday will open it up to a whole new audience

more on that at https://t.co/IhHDSxhYZS

#reinvent

@marknca tweeted at 01-Dec-2021, 17:13

lots of Redshift stuff. @awscloud knows that #serverless feature will intro a new audience to it, time to catch them up

#reinvent

@marknca tweeted at 01-Dec-2021, 17:15

finally an #ml tie-in!

#reinvent

@marknca tweeted at 01-Dec-2021, 17:16

…for those of you just joining this keynote at @awscloud #reinvent, this IS the #ml keynote

to be fair, 99% of ML is data clean up, organization, and management 😉

@marknca tweeted at 01-Dec-2021, 17:17

shifting the conversation to BI…QuickSight!!! :-)

#reinvent

@marknca tweeted at 01-Dec-2021, 17:18

more on QuickSight Q at https://t.co/uGnr57F5dk

#reinvent

@marknca tweeted at 01-Dec-2021, 17:18

if you’re wondering why I ❤️ QuickSight so much, it’s because it opens up a lot of analytical power to a wide audience within your business

that’s amazing powerful and something we don’t consider often enough

#reinvent

@marknca tweeted at 01-Dec-2021, 17:19

…also QuickSight has some very powerful sharing built in: https://docs.aws.amazon.com/quicksight/latest/user/sharing-a-dashboard.html

#reinvent

@marknca tweeted at 01-Dec-2021, 17:19

. @SwamiSivasubram’s on to @awscloud Glue while I’ve been ranting about QuickSight

my favourite AWS name, “AWS Glue Data Brew” << it’s true, you can use it too

#reinvent

@marknca tweeted at 01-Dec-2021, 17:20

more on @awscloud Glue Data Brew: https://t.co/C5YwVftsNO

#reinvent

@marknca tweeted at 01-Dec-2021, 17:20

Neeraja up now to talk about @awscloud Redshift #serverless + QuickSight

#reinvent

@marknca tweeted at 01-Dec-2021, 17:21

#reinvent

@marknca tweeted at 01-Dec-2021, 17:21

walking through a use case example now…

first the informercial “on-premises” view 🤦🤣

#reinvent

@marknca tweeted at 01-Dec-2021, 17:22

then right into the “don’t worry about it” #serverless approach with the new Redshift service

#reinvent

@marknca tweeted at 01-Dec-2021, 17:23

Redshift’s live data sharing capabilities lets more teams access the data without having to move it anywhere

more on the sharing features at https://t.co/wtTwWCuJU2

#reinvent

@marknca tweeted at 01-Dec-2021, 17:24

QuickSight comes into play now for the data analysis at the last mile. it enables ALL teams (not just BI specialists) to query the data and draw insights

#reinvent

@marknca tweeted at 01-Dec-2021, 17:25

what hits home for me with this example is the low operational overhead. almost all of the work & cost is directly driving business value

that’s the goal of the @awscloud Well-Architected Framework & these types of services make it way easier to strike that balance

#reinvent

@marknca tweeted at 01-Dec-2021, 17:27

“You’ve got to start treating data like an organizational asset”, @SwamiSivasubram

#reinvent

@marknca tweeted at 01-Dec-2021, 17:28

we’ve moved on to culture. another customer example up, this time it’s @ADP

#reinvent

@marknca tweeted at 01-Dec-2021, 17:29

lots of setup in the customer story, key point so far is that @adp handles so much data it’s like it’s a major country on it’s own

#reinvent

@marknca tweeted at 01-Dec-2021, 17:32

. @adp story was about using data at scale to drive change. no real technology aspect (beyond, "we couldn't do this without @awscloud")

quick message from @slack now…

#reinvent

@marknca tweeted at 01-Dec-2021, 17:39

on to innovation now...

#reinvent

@marknca tweeted at 01-Dec-2021, 17:40

this slide just keeps getting bigger each year

#reinvent

@marknca tweeted at 01-Dec-2021, 17:40

we’re a hop, skip, and a jump away from just a board of single pixels with just the brand’s dominant colour

#reinvent

@marknca tweeted at 01-Dec-2021, 17:41

I like this stacked layer approach to explaining the @awscloud #ml service offerings

basically, you want to start at the top layer & stay there as long as possible.

when necessary, move to the SageMaker layer

…hope you never need the bottom layer 🤣😉

#reinvent

@marknca tweeted at 01-Dec-2021, 17:44

. @SwamiSivasubram’s going to start at the bottom layer

#reinvent

@marknca tweeted at 01-Dec-2021, 17:44

this is all low level stuff, that’s why I recommend staying away from it

super interesting from a math/engineering perspective but also, quite a ways away from business value

#reinvent

@marknca tweeted at 01-Dec-2021, 17:45

more from the low-level layer of the #ml stack on @awscloud

#reinvent

@marknca tweeted at 01-Dec-2021, 17:46

more on the Trn1: https://t.co/jS08tVI7eI

#reinvent

@marknca tweeted at 01-Dec-2021, 17:46

…if you stay at the top of the stack!

(though SageMaker is amazing)

#reinvent

@marknca tweeted at 01-Dec-2021, 17:47

it’s just that SageMaker required #ml specific knowledge. that’s ok if you have it. but the top layer requires near zero #ml knowledge

just hand the service something and get a result…

#reinvent

@marknca tweeted at 01-Dec-2021, 17:47

ok, back to the keynote, talking about @awscloud SageMaker now. it’s an entire suite of solutions working together

#reinvent

@marknca tweeted at 01-Dec-2021, 17:48

…more of a prefix than anything, SageMaker _________

#reinvent

@marknca tweeted at 01-Dec-2021, 17:48

another customer story, this time it’s Aurora (not the database)

they do self-driving vehicle tech

#reinvent

@marknca tweeted at 01-Dec-2021, 17:50

this Aurora: https://aurora.tech/

#reinvent

@marknca tweeted at 01-Dec-2021, 17:50

so far the customer story is about the promise of self-driving cars

#reinvent

@marknca tweeted at 01-Dec-2021, 17:52

#reinvent

@marknca tweeted at 01-Dec-2021, 17:53

this customer story is cool but I feel like it’s missing the mark. it’s not technical enough for most builders and too technical for non-builders. right in the squishy middle that leaves me wondering, “What am I supposed to take away from this?”

#reinvent

@marknca tweeted at 01-Dec-2021, 17:57

I would love to get into the details of their #ml workflow or their strategy of overlaying models to deliver that self-driving vehicle

this is just an ad for the fact they are working on the tech. I wanted more

#reinvent

@marknca tweeted at 01-Dec-2021, 17:58

. @SwamiSivasubram is back up now. thankfully his energy will put this back on track. he’s still rolling strong…

#reinvent

@marknca tweeted at 01-Dec-2021, 17:58

structured data, the “easy” data type, is about 20% of the data #ml is working with

unstructured data is far more common

SageMaker has features to help with both

#reinvent

@marknca tweeted at 01-Dec-2021, 17:59

more on Amazon SageMaker Data Wrangler: https://t.co/bSWdnkkIVk

#reinvent

@marknca tweeted at 01-Dec-2021, 18:00

more on Amazon SageMaker Ground Truth at https://t.co/TMazhaB5lq

#reinvent

@marknca tweeted at 01-Dec-2021, 18:00

NEW FEATURE: Amazon SageMaker Ground Truth Plus << high quality training dataset fast, and reduce data labelling cost

more at https://t.co/pNpwc746kf

#reinvent

@marknca tweeted at 01-Dec-2021, 18:01

anything that helps with data labelling and clean up is very, very welcome

#reinvent

@marknca tweeted at 01-Dec-2021, 18:02

NEW FEATURE: Amazon SageMaker Studio Notebook << perform data engineering, analytics, and #ML workflows in one notebook.

GA today

#reinvent

@marknca tweeted at 01-Dec-2021, 18:04

we’re cruising right along with the #ml keynote at @awscloud #reinvent

@marknca tweeted at 01-Dec-2021, 18:05

NEW FEATURES (under the hood):

• Amazon SageMaker Training Compiler << 50% speed boost for training
• …Inference Recommender << reduce time to deploy
• … #Serverless Inference << f—k infrastructure 😉

#reinvent

@marknca tweeted at 01-Dec-2021, 18:07

that’s 5(?) services or features that have made a #serverless model available to users. I ❤️ it!

should be the default for everything…

#reinvent

@marknca tweeted at 01-Dec-2021, 18:08

if you didn’t see the announcement about SageMaker Canvas in @aselipsky’s keynote, you can read more at https://t.co/D0ggeQdxyg

#reinvent

@marknca tweeted at 01-Dec-2021, 18:08

Allie up now to walk us through SageMaker Canvas

#reinvent

@marknca tweeted at 01-Dec-2021, 18:09

the example is forecasting. something a lot of us do

#reinvent

@marknca tweeted at 01-Dec-2021, 18:09

the example is a model that forecasts the need for new computers for your team (a/k/a onboarding new ppl)

3 steps:

• access & browse the data
• prepare the data
• train & build the model

#reinvent

@marknca tweeted at 01-Dec-2021, 18:10

so, the example is covering all computer demands, not just onboarding

#reinvent

@marknca tweeted at 01-Dec-2021, 18:11

drag & drop various data sources to create one data set. SageMaker Canvas also tries to automatically clean up the data

…slick

#reinvent

@marknca tweeted at 01-Dec-2021, 18:12

what, I have to click?!? ugh.

🤣🤣🤣

@marknca tweeted at 01-Dec-2021, 18:13

the platform let’s you work through “what if” scenarios as well. that’s really powerful

#reinvent

@marknca tweeted at 01-Dec-2021, 18:13

the real question is how broadly applicable is this beyond forecasting. how will @awscloud SageMaker Canvas handle those other use cases?

can’t wait to find out!

#reinvent

@marknca tweeted at 01-Dec-2021, 18:14

. @SwamiSivasubram back up again now...

#reinvent

@marknca tweeted at 01-Dec-2021, 18:15

feels like we’re in the industrial #ml solutions now

https://t.co/0XhONuQnwa

#reinvent

@marknca tweeted at 01-Dec-2021, 18:16

tweeted too soon, that was the intro to the high level view of #ml solutions

#reinvent

@marknca tweeted at 01-Dec-2021, 18:16

quiet service callback, @awscloud Kendra, https://t.co/XIOdabtgx0

#reinvent

@marknca tweeted at 01-Dec-2021, 18:17

Kendra is one of several services that most builders forget even exists. that’s ok

that’s actually kind of cool. the @awscloud has grown to the point where there’s entire categories you’ll never touch as a builder

#reinvent

@marknca tweeted at 01-Dec-2021, 18:18

NEW FEATURE: Amazon Kendra Experience Builder << no code search application builder

#reinvent

@marknca tweeted at 01-Dec-2021, 18:19

more on EMR cluster and Spark job management with SageMaker Studio: https://t.co/tfGu3zUU7x

#reinvent

@marknca tweeted at 01-Dec-2021, 18:19

more on SageMaker’s new training compiler: https://t.co/AF3dkaYC4X

#reinvent

@marknca tweeted at 01-Dec-2021, 18:20

more on @awscloud SageMaker inference recommender: https://t.co/mvsaE99b51

#reinvent

@marknca tweeted at 01-Dec-2021, 18:20

. @SwamiSivasubram’s talking about @awscloud Connect and Lex experiences

#reinvent

@marknca tweeted at 01-Dec-2021, 18:21

NEW FEATURE: Amazon Lex Automated Chatbot Designer << simplifies bot design with natural language understanding

#reinvent

@marknca tweeted at 01-Dec-2021, 18:22

the Chatbot Designer sounds pretty cool. hopefully it’ll reduce user frustration

#reinvent

@marknca tweeted at 01-Dec-2021, 18:23

#reinvent

@marknca tweeted at 01-Dec-2021, 18:23

. @SwamiSivasubram talking about bringing more builders under the #ml umbrella. lots of work to be done in this area

#reinvent

@marknca tweeted at 01-Dec-2021, 18:24

NEW SERVICE: Amazon SageMaker Studio Lab << no cost, no setup #ml learning environment

sign up with an email address and start learning!

#reinvent

@marknca tweeted at 01-Dec-2021, 18:25

👆 this is huge! it should make it significantly easier to learn #ml technologies

goes hand-in-hand with the D2L textbook: https://d2l.ai/

#reinvent

@marknca tweeted at 01-Dec-2021, 18:26

NEW PROGRAM: @awscloud AI & ML Scholarship Program

#reinvent

@marknca tweeted at 01-Dec-2021, 18:30

more on @awscloud SageMaker Studio Lab at https://aws.amazon.com/blogs/aws/now-in-preview-amazon-sagemaker-studio-lab-a-free-service-to-learn-and-experiment-with-ml/

#reinvent

@marknca tweeted at 01-Dec-2021, 18:31

the lab is up at https://t.co/BgjK0AWsfZ

#reinvent

@marknca tweeted at 01-Dec-2021, 18:31

on to the @awscloud DeepRacer finale now. good luck to all!

#reinvent

/🧵

@marknca tweeted at 01-Dec-2021, 18:32

👆 Start