AWS offers a wide range of databases but making sense of these services can be tricky. In this live stream, we explore these data services and why you pick one over another.
Here are the slides that I used during the live stream.
Reasonably Accurate 馃馃 Transcript
All right. Good morning, everybody. How you doing? Thanks for jumping on the stream was quiet last week on streams not quiet work lies turn my position which is fantastic, but that kept me slightly busy last week. So I wasn鈥檛 able to jump on the streams. Let me just double check the quality.
He was lookin good. We are five by five awesome or live on Periscope through Twitter as well as the easiest place for comments. So I鈥檝e got it up. I can see it as we鈥檙e going along and showing you other things. So if you have some questions as I鈥檓 going along as absolutely the best time to ask them I will cover them after the fact is well.
I鈥檓 kind of new to see if other people out of traditional comments over time, but definitely ask them here in the Stream. I will do my best I鈥檓 to fight Them Off from Twitter as well at Twitter should be a little bit better. But of course I turned off notification and because I didn鈥檛 want to stream to get Disturbed.
It鈥檚 one of those sort of Catch-22. So picturesque fall graphic for cuz I certainly don鈥檛 have any leaves left on the trees around my neck of the woods. It was a beautiful fall, but it is a quickly is switching into winter, which is sort of a Halloween tradition up here where I live in Canada with every kid comes dressed as the same thing something underneath the park cuz it鈥檚 cold and sleep didn鈥檛 I see rain or snow starting off on the day? So we鈥檙e going to talk about is ews database service offerings.
So, let鈥檚 Dive Right In. Let me share my screen. We are going to go Google Chrome. So let鈥檚 make that a little bit bigger. So you guys can see it. So there is actually a database page for a tab us now they make this a little bit easier databases.
Here鈥檚 what we鈥檝e got to offer you the countdown clock to because this is a key part of the AWS offerings. And in fact, what we see is Vernor will get up on stage and talk about how they are expanding their database offerings and all of these different services that give you in a purpose-built databases, which is relatively true is a little bit of stretching in there.
But I have a wide variety far more than most other Cloud providers and they are all really high and which is great that they do have some differences and that is absolutely important to understand so they have a breakdown on this webpage. So this is if we look we go to Amazon AWS at amazon.com product sucks databases.
Rise and set the official database paid for these actually has a whole bunch of different use cases position on this page, which is great, but they all bubble down to storing data and being able to query it. So we already saw a very unique database structure and when we did the episode on streaming a three-hour ride the streaming episode when we talked about S3, then we talked about data as well and treated like a really unique applications.
But today we鈥檙e talking sort of more traditional old-school database Concepts and end with that end. Let鈥檚 start looking at this. So this is a set of slides I worked out for today wanted to walk you guys through this. There are four major will three major offerings in a tulle under the category of relational databases? He would traditionally think of as a database they store data in a table and you can query them out.
So in this case, we have Amazon Aurora, which is a truly cloud-native scalable a ridiculously low cost really really high and relational database service and you know it by default you should just be on Aurora and there are post grass and mice equal access is into Aurora and there is even a serverless version / feature in Aurora to make it easier to access from a divorce Lambda and service designs, but realistically if you鈥檙e doing a traditional database structure Aurora is where you want to start that is the most logical place.
It lines up sort of that best balance of I鈥檓 comfortable dealing with traditionalist. I get it out through two very popular standardized open-source interfaces postgres in my Sequel and it鈥檚 really you takes advantage that the fault tolerance in the cloud the cloud it is a managed. Service, which is great Amazon redshift is their database Warehouse.
So sort of old school database Warehouse application at rehearsals are the batch database processing is a great call men RTS is sort of the catch-all for everything else if you need a my Ms. SQL database if you need an Oracle database traditional postgres or traditional my sequel or Maria DB this is where you get it.
I鈥檓 so startled us migration service will let you move between any of these three other services but also importantly on-premise stuff. So if you have a database on-premise and that鈥檚 running like Microsoft SQL, you can actually sing your date of your triggers are your functions all that stuff over into like Aurora and it鈥檒l do the conversions as necessary.
Used in order to move data from production from intestine to production have you really want to be careful moving production data ended up in test at but very useful to migrate from a can pretty much be anywhere. So that鈥檚 traditional relational database documentdb spaced and is fully mongodb compatible.
An essential you鈥檙e storing documents in Korean based on any property in those documents. So a document in this case is a collection of key-value pairs. And so you can have you no title equals just databases author Eagles content and style of database where it鈥檚 very much off to the sort of up to you is the user to Define what goes in there traditionally, they don鈥檛 have schemas which is a data definition and then you can query as is very handy very useful in a lot of scenarios.
Especially too much is a graph database and graph databases really good for tracking relationships. Now, I know what you鈥檙e going to say. Category yeah, we did graphs are actually far better to track relationships and properties of according to those relationships and I鈥檒l show you why in detail in a second and the Amazon Neptune is the offering for me to get us there.
I was a little bit disappointed in that it鈥檚 a traditional sort of instance based offering so you still have to stand up an instant save a document d v I would like to see these be moved more into the serverless world, but you know baby steps. We鈥檙e getting there a lot of stuff in this column is a ledger database.
Please don鈥檛 even get me started on this. I guess I have to Define it fine. It鈥檚 a blockchain based Ledger database. It鈥檚 called the Amazon Quantum Ledger database. It鈥檚 best if we just move on I am extremely Limited in its use cases. If you have one of these use cases extremely useful it is immutable ledger so you can鈥檛 change things things sort of like a double-entry accounting.
You can鈥檛 remove something once you put it in there. You need to make a counter on Entry have a lot of stuff. It doesn鈥檛 belong but we鈥檒l talk another time at time series the new last year reinvent along with Neptune was Amazon timestream. That is an extremely powerful database if you are data is based on a Time series.
So you鈥檙e tracking things like events. If you are a if everything has a timestamp to it time series might be where you want to go depending on the type of what you want to do with that data. Finally. We have key value. Amazon dynamodb, this is going to be the number one workhorse outside of relational databases for most of us and we鈥檒l actually dive into that console in a minute.
And I though it鈥檚 been a long time since I played it so bear with me but yeah, we鈥檒l get some hands-on experience in this very very strong fast very very powerful if you want an in-memory database. So that鈥檚 the breakdown of the database offerings. Which one do I use the number one thing you need to think about this while there鈥檚 two stages is basically think about are you going to be if you鈥檙e and then based on that next question, how are you pulling data out? Like what do you want to do with this data? And that鈥檚 absolutely critical to documentdb received.
A lot of the time for mobile applications. We need to send a chunk of related information to a mobile client with one query. So you鈥檙e trying to reduce Network back and forth so you to say hey give me today鈥檚 episode metadata and there you get it all in one query with one quick look up a relational database is been around for 40 plus years.
It鈥檚 a well-known as far as it鈥檚 it鈥檚 sort of edge cases and how best to design and set things up. But yeah taking your data out of the absolute worse or the most critical thing to think about in that really will tell you what you鈥檙e going to be picking.
So I would highly recommend spending more time writing data are writing actions on your side. That put more work on you to write things in a data structure that makes it easier to pull out because that鈥檚 the majority of most is actually clearing the data and pulling it out.
So let me walk you through a very simple example, let鈥檚 look at a traditional relational database set up. So he so I鈥檓 there a bunch of fake names are there we have buses. So these would be a busser. It鈥檚 a bus route one would go from the airport to the hotel bus route goes from the hotel to the office, which is great.
And then we have a table that helps connect these so to say who is written on what bus so if we will get my ID my name ideas one if we look at the buses bus to so I went from the hotel to the office Name ID one that鈥檚 me again Road bus one.
I went to the airport to the hotel, right? So that鈥檚 how we connect these two pieces of Information Tracking and the names table on tracking all the people. Know about all my bus riders in the buses table and tracking all my bus routes and then I created this new table called names to buses that links these two to make it possible to query to say how many buses which passes did Mark take or which writers were on this bus you can go either way.
And what ends up happening is you write a query that looks kind of like this Rites of issus standard relational database query written in SQL is double check for the words as well. But if we have our standard relational database query, it does break down to someone English is at it鈥檚 not that hard to understand.
I think when you break it down so select. So give me Star is everything from names. Okay, that makes sense. Give me everything from names and I鈥檓 going to call and and names and just to make it easier. So instead of typing up names. I just use the with the letter n Inner join with me in Schenectady Sue tables names the buses and I鈥檒l call that and to be on and ID so Name ID and names the buses Name ID right that make sense.
I鈥檓 looking up the same thing. So I鈥檓 saying in table names. There鈥檚 a field called ID in names the buses. There is a field called Name ID the same connect those two tables all the values in those two tables. Furthermore. Here鈥檚 another inner join furthermore connected table buses on the bus ID.
When it鈥檚 equal two names two buses bus ID that name ID in the name of the buses table and then also grab me the buses information when the bus ID equals the same bus that鈥檚 in the names of the buses. So what that effectively does is connects these three table by saying when Mark Donna covin, so when I have an idea of one here in my name鈥檚 the buses, I have an idea of one and then a bus idea to grab that right? And then for the second row, I鈥檓 going to get another entry where it鈥檚 marked on the Cove in 121 airport hotel.
So there will be in the result of this one for each of the bus rides. I took that is almost standard literally textbook SQL and it鈥檚 clunky is all good at it works. But if you start to expand Beyond Relationships are this is a second-order relationship. We have or first order relationships are we have names to buses? And that鈥檚 what we鈥檙e connecting you start to do a second and third order relationships or & Order relationship to get really bad really fast.
You get these monsters queries, you make these tables more than these kind of names the buses tables of these odd connection tables and yes, it is a relational database, but I think if you鈥檝e been building anything for any length of time, you know that database technology like this gets we take this concept this exact same concept.
We鈥檙e going to look at how a graph database wood store its own this case 8 Amazon Neptune. We鈥檝e got the exact same information here, but I think you was a user understand this far better, right? We have our Riders on the outside in the blue balls. And in the in the green circles, we鈥檝e got our buses right to Blue circles.
Riders this week and then I in the green circles, we鈥檝e got our bus and then we have relationships know all these lines. I just colored them differently make it easier to see them but you see we have a relationship and because I鈥檝e only done it once or twice Mark rides the Number 2 bus but it鈥檚 a big thick line because I ride that bus far more often same with Jen in the opposite corner.
She rides the bus the first bus quite a bit more then she writes the second one or lean here only writes the one bus and Walter has moderate rides. And now this as a graph database it stores it in a whole bunch of really cool data structures in the back and that you don鈥檛 have to worry about conceptually.
You just need to think about this kind of graph structure that you see on your on the screen right now on this slide. Now, we want to the syntax is going to change depending on the graph database that you鈥檙e working with but the concept is all the same. So you see on the right left hand side hear the buses this document structure.
Patrick is literally query and graphql. So I want bus and I want the names of all the Riders related to bus first and last that鈥檚 it. That鈥檚 a far simpler relationship a pole. Then you want then in a relational database write and graph can get really complicated really really quickly really understandable have felt his community hero Gemma Eileen Smith gave great talk at the public sector Summit in DC how she had helped organize the New York using open data from the city make relationship connections between a various actions.
So when there was like a spark spike in parking tickets were connected that to your 311 complaints and Construction in the area and this far simpler all store in a graph database, right? Because graph databases are all about relationships and it seems very backwards. Compared to a relational database and a graph database to connect to other tables to make multiple relationships between them.
Where is the graph database? And I forgive the rapid scrolling hear the graph database structure. This is how you would draw doubt if I said draw this out on a on a whiteboard tell me who takes what bus would Draw Something far more similar to this rather than Table after Table after table of information.
So this is just a quick illustration of why you would pick one database over the other if we鈥檙e trying to figure out bus ridership in an area a graph database makes way more sense, especially if we start to add an additional data points like places. So if we start to add in places, where did they get off on these bus routes, right? We could add in a Place Circle for the office and then we can quickly see while how many who Getting off their which bus routes were getting more service to each of these places when there was multiple routes in place and it just a far simpler way to understand.
So that鈥檚 8 Amazon Neptune, which is what I come up with a serverless version of it because I don鈥檛 want to manage instances, but that is a core difference between those types of databases and you have to keep asking those similar questions when you鈥檙e looking. Sorry to hear we got when you鈥檙e looking across these different options.
Which one do you pick? It really depends on how you want to pull data out? What do you want to do with that data, right. You never going to pick something like Amazon elasticache for long-term care is what you鈥檙e going to pick up to speed up, and a shove things into sits in front these in-memory databases tend to sit in front of another database a lot of the time he values what you actually want when you鈥檙e trying relational database in Amazon DB makes it really easy to do.
Bowser my screen so if I go to dynamodb options myself in a Management console and flexible, no SQL database service you create tables at a very straightforward. So we鈥檙e going to create a table. We鈥檙e going to call this. Let鈥檚 go. Same data buses. The primary key is bus ID and it鈥檚 going to be a number right? So we鈥檙e also going to have to sort key which is going to be a place.
So we have a table in buses leave a bus ID. We are adding a sort key. That鈥檚 our primary key or sorc he is also going to be place. So we鈥檙e in a sort out by that and then we鈥檙e going to use default index now, I don鈥檛 have auto-scaling enabled by default.
That鈥檚 a relatively new dynamodb service hours feature and you really should enable that we don鈥檛 have time to go through the documentation right now. Basically, what it allows you to do is let a dynamodb throttle things for you automatically. That鈥檚 fine. We鈥檙e going to just click the blue create button and this is going to create our table so it鈥檚 being created right now.
Which is great. So the idea here is that you don鈥檛 want to actually end up connecting tables like we did in the SQL. So if we look our overview, which has been created we don鈥檛 have any streams. That鈥檚 fine. We have no items so we can start to add items.
We could also add. So here we got a bus ID. We鈥檝e got a place we can add our bus ID is 1 places office. We can actually add another. second stop attribute value is going to be Hotel. So we can start creating our items like this standard sort of relational database, but you can start to get really complicated and actually add connection so you can set it up like we did with the traditional database so I can create a second table.
I can create my name鈥檚 table. And my name is going to be first and I鈥檒l add a sort Key by last so that鈥檚 going to create a second one. And what you can do here is because of the way this works now forgive me. If I am breaking a lot of Dynamo rules.
I probably am I am not as up to speed on my Dynamo optimize structures that I should be. But if I take this I can add a binary what鈥檚 bad a binary set and I鈥檓 going to add rides and in this case what I鈥檓 going to do. There we go.
And then I sat a second ride. You鈥檙e probably wondering what I鈥檓 doing what I鈥檓 doing here. No or empty values, what are more parameter values are in physical of live streams, right and then within that record directly, I鈥檓 going to get that relationship of how many bus rides are taken.
So I鈥檓 going to keep a binary set which is just don鈥檛 want to buy that鈥檚 the problem. What I want is a number set on a binary set. Or a map and see if I can get a map. Thanks. Let me kill list. Remove append number set my apologies there.
Too perfect. So say that has down by a table to overall optimize this so instead of creating multiple tables and I have buses. So I got a bus ID in a place in stop and but also in my right or record, I am keeping track of rides and that is because most of time I want to know where my Riders at what write multiple Riders are taking some more boxes so I can combine routes maybe later at but also possible to query based on ride the ride values to then line up to see who鈥檚 taking the bus has far more performing this way in again.
There鈥檚 probably a six different ways to design this. I鈥檓 in a different structure and but if I want a query this I can do the same thing. I can write a simple query that goes across these two tables and essentially say you don鈥檛 want for Ryder Mark. I want to see what the bus of rides are at.
What I would do when I鈥檓 building an application as I would actually just query the buses cuz I have a finite number for whatever. Bus routes than Riders devices keep that in memory and then just say Marco KU Road bus to what鈥檚 that one of us one? Okay, great with that one and then you could also because this is a set you can work over the satin say how many times have I taken bus number to so if I change this record again, if I go into my rides right onto another ride onto so now I鈥檝e taken multiple buses courses duplicates because I鈥檓 not setting this up correctly and but that kind of concept Works similarly, right? So very rough example of how to use Dynamo.
I鈥檓 simply because I鈥檓 out of practice which I apologize for what you get the concept right? We walked through. Let me share that again real fast and we walk through some of the and of course I closed it. We walked through some of the different types of offerings of databases.
So if you鈥檙e looking for relation, So I guess you鈥檙e setting up and tables you want to be sitting in Aurora at the document documentdb those graphs. If you鈥檙e thinking of things like this graph database Amazon Neptune is the way to go time streams really intriguing to me. Actually. I am I don鈥檛 think it gets enough play make it still in its early stages but a lot of stuff that gets shoved into a relational databases really time series data.
So if you have something that鈥檚 explicitly time series in your thinking about it in the concept of Time techno timestream, a lot of cloud native designs actually end up in dynamodb. Do not let my fumbling around it dissuade you it is the best database service out there. I think especially need to be as Cloud.
It is quite powerful. There are some phenomenal courses out there on dynamodb some great information some people much better than I at At it and it鈥檚 a great reminder that I just need to get my hands dirty little bit more. I need to dive in. So I that鈥檚 a quick overview database Services.
Appreciate you joining me here on LinkedIn. I appreciate you joining me on Twitter. Hopefully, this has been useful. Please let me know. If you have any questions at what you鈥檇 like to see in the next stream. It鈥檚 probably going to be next week. I got a bunch of stuff to do for reinvent this week.
So trying to get that out and I always appreciate your time and let me know what you think. Will talk to you soon. Thanks for joining. All right. Good morning, everybody. How you doing? Thanks for jumping on the stream was quiet last week on streams not quiet work lies turn my position which is fantastic, but that kept me slightly busy last week.
So I wasn鈥檛 able to jump on the streams. Let me just double check the quality. He was lookin good. We are five by five awesome or live on Periscope through Twitter as well as the easiest place for comments. So I鈥檝e got it up. I can see it as we鈥檙e going along and showing you other things.
So if you have some questions as I鈥檓 going along as absolutely the best time to ask them I will cover them after the fact is well. I鈥檓 kind of new to see if other people out of traditional comments over time, but definitely ask them here in the Stream. I will do my best I鈥檓 to fight Them Off from Twitter as well at Twitter should be a little bit better.
But of course I turned off notification and because I didn鈥檛 want to stream to get Disturbed. It鈥檚 one of those sort of Catch-22. So picturesque fall graphic for cuz I certainly don鈥檛 have any leaves left on the trees around my neck of the woods. It was a beautiful fall, but it is a quickly is switching into winter, which is sort of a Halloween tradition up here where I live in Canada with every kid comes dressed as the same thing something underneath the park cuz it鈥檚 cold and sleep didn鈥檛 I see rain or snow starting off on the day? So we鈥檙e going to talk about is ews database service offerings.
So, let鈥檚 Dive Right In. Let me share my screen. We are going to go Google Chrome. So let鈥檚 make that a little bit bigger. So you guys can see it. So there is actually a database page for a tab us now they make this a little bit easier databases.
Here鈥檚 what we鈥檝e got to offer you the countdown clock to because this is a key part of the AWS offerings. And in fact, what we see is Vernor will get up on stage and talk about how they are expanding their database offerings and all of these different services that give you in a purpose-built databases, which is relatively true is a little bit of stretching in there.
But I have a wide variety far more than most other Cloud providers and they are all really high and which is great that they do have some differences and that is absolutely important to understand so they have a breakdown on this webpage. So this is if we look we go to Amazon AWS at amazon.com product sucks databases.
Rise and set the official database paid for these actually has a whole bunch of different use cases position on this page, which is great, but they all bubble down to storing data and being able to query it. So we already saw a very unique database structure and when we did the episode on streaming a three-hour ride the streaming episode when we talked about S3, then we talked about data as well and treated like a really unique applications.
But today we鈥檙e talking sort of more traditional old-school database Concepts and end with that end. Let鈥檚 start looking at this. So this is a set of slides I worked out for today wanted to walk you guys through this. There are four major will three major offerings in a tulle under the category of relational databases? He would traditionally think of as a database they store data in a table and you can query them out.
So in this case, we have Amazon Aurora, which is a truly cloud-native scalable a ridiculously low cost really really high and relational database service and you know it by default you should just be on Aurora and there are post grass and mice equal access is into Aurora and there is even a serverless version / feature in Aurora to make it easier to access from a divorce Lambda and service designs, but realistically if you鈥檙e doing a traditional database structure Aurora is where you want to start that is the most logical place.
It lines up sort of that best balance of I鈥檓 comfortable dealing with traditionalist. I get it out through two very popular standardized open-source interfaces postgres in my Sequel and it鈥檚 really you takes advantage that the fault tolerance in the cloud the cloud it is a managed. Service, which is great Amazon redshift is their database Warehouse.
So sort of old school database Warehouse application at rehearsals are the batch database processing is a great call men RTS is sort of the catch-all for everything else if you need a my Ms. SQL database if you need an Oracle database traditional postgres or traditional my sequel or Maria DB this is where you get it.
I鈥檓 so startled us migration service will let you move between any of these three other services but also importantly on-premise stuff. So if you have a database on-premise and that鈥檚 running like Microsoft SQL, you can actually sing your date of your triggers are your functions all that stuff over into like Aurora and it鈥檒l do the conversions as necessary.
Used in order to move data from production from intestine to production have you really want to be careful moving production data ended up in test at but very useful to migrate from a can pretty much be anywhere. So that鈥檚 traditional relational database documentdb spaced and is fully mongodb compatible.
An essential you鈥檙e storing documents in Korean based on any property in those documents. So a document in this case is a collection of key-value pairs. And so you can have you no title equals just databases author Eagles content and style of database where it鈥檚 very much off to the sort of up to you is the user to Define what goes in there traditionally, they don鈥檛 have schemas which is a data definition and then you can query as is very handy very useful in a lot of scenarios.
Especially too much is a graph database and graph databases really good for tracking relationships. Now, I know what you鈥檙e going to say. Category yeah, we did graphs are actually far better to track relationships and properties of according to those relationships and I鈥檒l show you why in detail in a second and the Amazon Neptune is the offering for me to get us there.
I was a little bit disappointed in that it鈥檚 a traditional sort of instance based offering so you still have to stand up an instant save a document d v I would like to see these be moved more into the serverless world, but you know baby steps. We鈥檙e getting there a lot of stuff in this column is a ledger database.
Please don鈥檛 even get me started on this. I guess I have to Define it fine. It鈥檚 a blockchain based Ledger database. It鈥檚 called the Amazon Quantum Ledger database. It鈥檚 best if we just move on I am extremely Limited in its use cases. If you have one of these use cases extremely useful it is immutable ledger so you can鈥檛 change things things sort of like a double-entry accounting.
You can鈥檛 remove something once you put it in there. You need to make a counter on Entry have a lot of stuff. It doesn鈥檛 belong but we鈥檒l talk another time at time series the new last year reinvent along with Neptune was Amazon timestream. That is an extremely powerful database if you are data is based on a Time series.
So you鈥檙e tracking things like events. If you are a if everything has a timestamp to it time series might be where you want to go depending on the type of what you want to do with that data. Finally. We have key value. Amazon dynamodb, this is going to be the number one workhorse outside of relational databases for most of us and we鈥檒l actually dive into that console in a minute.
And I though it鈥檚 been a long time since I played it so bear with me but yeah, we鈥檒l get some hands-on experience in this very very strong fast very very powerful if you want an in-memory database. So that鈥檚 the breakdown of the database offerings. Which one do I use the number one thing you need to think about this while there鈥檚 two stages is basically think about are you going to be if you鈥檙e and then based on that next question, how are you pulling data out? Like what do you want to do with this data? And that鈥檚 absolutely critical to documentdb received.
A lot of the time for mobile applications. We need to send a chunk of related information to a mobile client with one query. So you鈥檙e trying to reduce Network back and forth so you to say hey give me today鈥檚 episode metadata and there you get it all in one query with one quick look up a relational database is been around for 40 plus years.
It鈥檚 a well-known as far as it鈥檚 it鈥檚 sort of edge cases and how best to design and set things up. But yeah taking your data out of the absolute worse or the most critical thing to think about in that really will tell you what you鈥檙e going to be picking.
So I would highly recommend spending more time writing data are writing actions on your side. That put more work on you to write things in a data structure that makes it easier to pull out because that鈥檚 the majority of most is actually clearing the data and pulling it out.
So let me walk you through a very simple example, let鈥檚 look at a traditional relational database set up. So he so I鈥檓 there a bunch of fake names are there we have buses. So these would be a busser. It鈥檚 a bus route one would go from the airport to the hotel bus route goes from the hotel to the office, which is great.
And then we have a table that helps connect these so to say who is written on what bus so if we will get my ID my name ideas one if we look at the buses bus to so I went from the hotel to the office Name ID one that鈥檚 me again Road bus one.
I went to the airport to the hotel, right? So that鈥檚 how we connect these two pieces of Information Tracking and the names table on tracking all the people. Know about all my bus riders in the buses table and tracking all my bus routes and then I created this new table called names to buses that links these two to make it possible to query to say how many buses which passes did Mark take or which writers were on this bus you can go either way.
And what ends up happening is you write a query that looks kind of like this Rites of issus standard relational database query written in SQL is double check for the words as well. But if we have our standard relational database query, it does break down to someone English is at it鈥檚 not that hard to understand.
I think when you break it down so select. So give me Star is everything from names. Okay, that makes sense. Give me everything from names and I鈥檓 going to call and and names and just to make it easier. So instead of typing up names. I just use the with the letter n Inner join with me in Schenectady Sue tables names the buses and I鈥檒l call that and to be on and ID so Name ID and names the buses Name ID right that make sense.
I鈥檓 looking up the same thing. So I鈥檓 saying in table names. There鈥檚 a field called ID in names the buses. There is a field called Name ID the same connect those two tables all the values in those two tables. Furthermore. Here鈥檚 another inner join furthermore connected table buses on the bus ID.
When it鈥檚 equal two names two buses bus ID that name ID in the name of the buses table and then also grab me the buses information when the bus ID equals the same bus that鈥檚 in the names of the buses. So what that effectively does is connects these three table by saying when Mark Donna covin, so when I have an idea of one here in my name鈥檚 the buses, I have an idea of one and then a bus idea to grab that right? And then for the second row, I鈥檓 going to get another entry where it鈥檚 marked on the Cove in 121 airport hotel.
So there will be in the result of this one for each of the bus rides. I took that is almost standard literally textbook SQL and it鈥檚 clunky is all good at it works. But if you start to expand Beyond Relationships are this is a second-order relationship. We have or first order relationships are we have names to buses? And that鈥檚 what we鈥檙e connecting you start to do a second and third order relationships or & Order relationship to get really bad really fast.
You get these monsters queries, you make these tables more than these kind of names the buses tables of these odd connection tables and yes, it is a relational database, but I think if you鈥檝e been building anything for any length of time, you know that database technology like this gets we take this concept this exact same concept.
We鈥檙e going to look at how a graph database wood store its own this case 8 Amazon Neptune. We鈥檝e got the exact same information here, but I think you was a user understand this far better, right? We have our Riders on the outside in the blue balls. And in the in the green circles, we鈥檝e got our buses right to Blue circles.
Riders this week and then I in the green circles, we鈥檝e got our bus and then we have relationships know all these lines. I just colored them differently make it easier to see them but you see we have a relationship and because I鈥檝e only done it once or twice Mark rides the Number 2 bus but it鈥檚 a big thick line because I ride that bus far more often same with Jen in the opposite corner.
She rides the bus the first bus quite a bit more then she writes the second one or lean here only writes the one bus and Walter has moderate rides. And now this as a graph database it stores it in a whole bunch of really cool data structures in the back and that you don鈥檛 have to worry about conceptually.
You just need to think about this kind of graph structure that you see on your on the screen right now on this slide. Now, we want to the syntax is going to change depending on the graph database that you鈥檙e working with but the concept is all the same. So you see on the right left hand side hear the buses this document structure.
Patrick is literally query and graphql. So I want bus and I want the names of all the Riders related to bus first and last that鈥檚 it. That鈥檚 a far simpler relationship a pole. Then you want then in a relational database write and graph can get really complicated really really quickly really understandable have felt his community hero Gemma Eileen Smith gave great talk at the public sector Summit in DC how she had helped organize the New York using open data from the city make relationship connections between a various actions.
So when there was like a spark spike in parking tickets were connected that to your 311 complaints and Construction in the area and this far simpler all store in a graph database, right? Because graph databases are all about relationships and it seems very backwards. Compared to a relational database and a graph database to connect to other tables to make multiple relationships between them.
Where is the graph database? And I forgive the rapid scrolling hear the graph database structure. This is how you would draw doubt if I said draw this out on a on a whiteboard tell me who takes what bus would Draw Something far more similar to this rather than Table after Table after table of information.
So this is just a quick illustration of why you would pick one database over the other if we鈥檙e trying to figure out bus ridership in an area a graph database makes way more sense, especially if we start to add an additional data points like places. So if we start to add in places, where did they get off on these bus routes, right? We could add in a Place Circle for the office and then we can quickly see while how many who Getting off their which bus routes were getting more service to each of these places when there was multiple routes in place and it just a far simpler way to understand.
So that鈥檚 8 Amazon Neptune, which is what I come up with a serverless version of it because I don鈥檛 want to manage instances, but that is a core difference between those types of databases and you have to keep asking those similar questions when you鈥檙e looking. Sorry to hear we got when you鈥檙e looking across these different options.
Which one do you pick? It really depends on how you want to pull data out? What do you want to do with that data, right. You never going to pick something like Amazon elasticache for long-term care is what you鈥檙e going to pick up to speed up, and a shove things into sits in front these in-memory databases tend to sit in front of another database a lot of the time he values what you actually want when you鈥檙e trying relational database in Amazon DB makes it really easy to do.
Bowser my screen so if I go to dynamodb options myself in a Management console and flexible, no SQL database service you create tables at a very straightforward. So we鈥檙e going to create a table. We鈥檙e going to call this. Let鈥檚 go. Same data buses. The primary key is bus ID and it鈥檚 going to be a number right? So we鈥檙e also going to have to sort key which is going to be a place.
So we have a table in buses leave a bus ID. We are adding a sort key. That鈥檚 our primary key or sorc he is also going to be place. So we鈥檙e in a sort out by that and then we鈥檙e going to use default index now, I don鈥檛 have auto-scaling enabled by default.
That鈥檚 a relatively new dynamodb service hours feature and you really should enable that we don鈥檛 have time to go through the documentation right now. Basically, what it allows you to do is let a dynamodb throttle things for you automatically. That鈥檚 fine. We鈥檙e going to just click the blue create button and this is going to create our table so it鈥檚 being created right now.
Which is great. So the idea here is that you don鈥檛 want to actually end up connecting tables like we did in the SQL. So if we look our overview, which has been created we don鈥檛 have any streams. That鈥檚 fine. We have no items so we can start to add items.
We could also add. So here we got a bus ID. We鈥檝e got a place we can add our bus ID is 1 places office. We can actually add another. second stop attribute value is going to be Hotel. So we can start creating our items like this standard sort of relational database, but you can start to get really complicated and actually add connection so you can set it up like we did with the traditional database so I can create a second table.
I can create my name鈥檚 table. And my name is going to be first and I鈥檒l add a sort Key by last so that鈥檚 going to create a second one. And what you can do here is because of the way this works now forgive me. If I am breaking a lot of Dynamo rules.
I probably am I am not as up to speed on my Dynamo optimize structures that I should be. But if I take this I can add a binary what鈥檚 bad a binary set and I鈥檓 going to add rides and in this case what I鈥檓 going to do. There we go.
And then I sat a second ride. You鈥檙e probably wondering what I鈥檓 doing what I鈥檓 doing here. No or empty values, what are more parameter values are in physical of live streams, right and then within that record directly, I鈥檓 going to get that relationship of how many bus rides are taken.
So I鈥檓 going to keep a binary set which is just don鈥檛 want to buy that鈥檚 the problem. What I want is a number set on a binary set. Or a map and see if I can get a map. Thanks. Let me kill list. Remove append number set my apologies there.
Too perfect. So say that has down by a table to overall optimize this so instead of creating multiple tables and I have buses. So I got a bus ID in a place in stop and but also in my right or record, I am keeping track of rides and that is because most of time I want to know where my Riders at what write multiple Riders are taking some more boxes so I can combine routes maybe later at but also possible to query based on ride the ride values to then line up to see who鈥檚 taking the bus has far more performing this way in again.
There鈥檚 probably a six different ways to design this. I鈥檓 in a different structure and but if I want a query this I can do the same thing. I can write a simple query that goes across these two tables and essentially say you don鈥檛 want for Ryder Mark. I want to see what the bus of rides are at.
What I would do when I鈥檓 building an application as I would actually just query the buses cuz I have a finite number for whatever. Bus routes than Riders devices keep that in memory and then just say Marco KU Road bus to what鈥檚 that one of us one? Okay, great with that one and then you could also because this is a set you can work over the satin say how many times have I taken bus number to so if I change this record again, if I go into my rides right onto another ride onto so now I鈥檝e taken multiple buses courses duplicates because I鈥檓 not setting this up correctly and but that kind of concept Works similarly, right? So very rough example of how to use Dynamo.
I鈥檓 simply because I鈥檓 out of practice which I apologize for what you get the concept right? We walked through. Let me share that again real fast and we walk through some of the and of course I closed it. We walked through some of the different types of offerings of databases.
So if you鈥檙e looking for relation, So I guess you鈥檙e setting up and tables you want to be sitting in Aurora at the document documentdb those graphs. If you鈥檙e thinking of things like this graph database Amazon Neptune is the way to go time streams really intriguing to me. Actually. I am I don鈥檛 think it gets enough play make it still in its early stages but a lot of stuff that gets shoved into a relational databases really time series data.
So if you have something that鈥檚 explicitly time series in your thinking about it in the concept of Time techno timestream, a lot of cloud native designs actually end up in dynamodb. Do not let my fumbling around it dissuade you it is the best database service out there. I think especially need to be as Cloud.
It is quite powerful. There are some phenomenal courses out there on dynamodb some great information some people much better than I at At it and it鈥檚 a great reminder that I just need to get my hands dirty little bit more. I need to dive in. So I that鈥檚 a quick overview database Services.
Appreciate you joining me here on LinkedIn. I appreciate you joining me on Twitter. Hopefully, this has been useful. Please let me know. If you have any questions at what you鈥檇 like to see in the next stream. It鈥檚 probably going to be next week. I got a bunch of stuff to do for reinvent this week.
So trying to get that out and I always appreciate your time and let me know what you think. Will talk to you soon. Thanks for joining.
