This post was originally written for
Trend Micro .Reasonably Accurate 馃馃 Transcript
[00:00:00] Mark: Hello, everybody. Welcome. Welcome back. For those of you that tried to join us this morning. If you did try to join us from this morning, um, you can see from the background already that we're back into the sort of traditional studio set up. Um, as opposed to trying to do it on site, we're actually, um, in the trend micro office in Ottawa this morning, um, on a mobile set up, had a ton of problems.
[00:00:34] I really appreciate your patience. Um, we learned some stuff. Um, Alright, every, every opportunity, uh, for, uh, every failure is very much an opportunity for learning. Um, I learned some stuff, uh, you know, we were doing the mobile setup, we had audio problems, we had lagging problems, we had bandwidth problems.
[00:00:50] And that, uh, should not have been a surprise to me, though it was. If you're in a building full of engineers Bandwidth is not something that is going to be plentiful because everyone's doing a ton of stuff. They're creating all [00:01:00] sorts of crazy stuff. Everybody's streaming stuff, pulling things down.
[00:01:02] There's just a lot of usage. We did not get enough bandwidth to get a good quality signal, which is why we pulled the plug after eight extremely awkward minutes on the stream, um, learning and, and appreciating your patience. Um, so thank you for jumping back on the stream. We are live on LinkedIn. We are live on YouTube.
[00:01:19] We are live on Twitter. We've got folks already tuned in from all around the world. I see folks from India, Kuwait, from the KSA, from Portugal, from Russia, a whole bunch of places in the EU, people tuning in from North America, South America. Thank you very, very much, especially after us. You'll notice in the title of this, this is a containers v1.
[00:01:38] 1. Um, because V1 was a failed build. It did not pass our integration tests. Um, yet we still tried to deploy, which is just, you know, we'll talk about it in a minute. Um, but if you are, uh, watching us here, uh, we are monitoring the comments on LinkedIn. We are monitoring on YouTube as well as Twitter. If you have questions, please post it, uh, there.
[00:01:57] We will try to address them as we go in the stream. [00:02:00] Today's conversation is about containers. Uh, we are going to tackle it all over. Um, every kind of angle we can think of. With that, I'm going to bring in our guest today. Uh, Jeff Baskwell is one of the smartest people I know. He is a phenomenal developer.
[00:02:17] Um, he is already super embarrassed, but now you're on the stream and you got to go with it cause I can't, you can't hang up. We already failed once trying technical. So I am just going to fire comments your way and compliments because if I say it, I mean it. Um, Jeff is a software developer. I tried to use his actual title, but he will not let me.
[00:02:36] So we're just saying, you know, software developer. Needless to say, it's very high up there and he's very, very good. Uh, Jeff, welcome to the stream. You want to say a little bit
[00:02:45] Geoff: about
[00:02:45] Mark: yourself?
[00:02:47] Geoff: Thanks, Mark. Thanks for the incredible intro. Um, So yeah, I'm, I'm Jeff Baskill. I've been doing software development for a long time, um, quite, quite a long time, um, [00:03:00] I don't actually want to say, uh, but, uh, I've been, I worked in, uh, telecoms for, uh, for probably about 20 years, and then I moved over to, uh, to Trend and started doing, uh, security work.
[00:03:11] And, uh, I've been working with, uh, containers for, I'm gonna say about two years now. Like, really actively working with them. Um, learned a lot. Learned that there's a lot more to learn. Um, and, yeah. So, I think that's a,
[00:03:27] Mark: maybe a good start. Yeah, that's great. That's fantastic. Very modest introduction. But, you know, I think people will get it as we, as we dive into this.
[00:03:34] So, you mentioned you've been working with containers, um, you know, really heavy for the last couple years.
[00:03:38] Geoff: Yep.
[00:03:39] Mark: Let's start that off then. Um, what, in your view, what are the, what's the point of containers? A lot of people hear the word containers. They know it's the coolness, you know, and there's a lot of conferences around it, a lot of talks around it.
[00:03:50] From your perspective, what's the point?
[00:03:54] Geoff: Um, let's see. So from my perspective, the point is, um, being able to take your [00:04:00] software that you've built And deploy it really easily, right? And not worry a whole lot about the environment that you're deploying it into. Um, in, in previous places, um, you had to worry about, okay, is, is the VM running the right operating system?
[00:04:18] Is, are the right dependencies in place? You know, shared libraries all over the place. You kind of worry about everything that's there. And with containers, you kind of package up your application, right? And it's just. It's a simple way to package it off. You say, here it is, it's exactly the same on my laptop as it is when it's deployed.
[00:04:37] Um, and that's, that's kind of the easiest thing about it is, um, and I mean, from my perspective, the most important thing about it is that you ship what you worked with on your computer and it's going to be the same, right? You don't have to worry about, uh, strange stuff changing underneath you. We'll get back to that.
[00:04:56] Mark: Almost caught yourself in a little trap there, didn't you? But I mean, I think [00:05:00] the point, you know, just from the way you were describing it, I'm going to go out on a limb and say, you've had that problem before in your career where you've had something working just fine in your environment. And when you try to push it out, it no longer works.
[00:05:12] Geoff: Yeah, absolutely. Um, that happens like. Probably the most often, um, that is one of the biggest problems that you run into when you're building software sort of in the traditional way where you're saying, okay, I'm just going to ship a zip file or I'm going to ship them and install it, right, and the customer brings their own OS and, uh, and you just sort of try to deal with, uh, whatever might be there, um, and kind of predict what might be there.
[00:05:38] Um, it's gotten, it was, it got better with, uh, you know, shipping VM images, but VM images are huge, right? And so containers let you just say, I'm going to take this small application, ideally a small application, and ship just that bit of it out to the customer. They can run it in the container environment and it just comes up.
[00:05:58] It's small, it's simple, it's [00:06:00] fast.
[00:06:01] Mark: And I think those are, those are really the three keys, right? Small, simple, and fast. Um, you know, and I mean, we're going to jump around a bit, but what kind of mistakes do you see people making? Because I mean, not only you're obviously an active developer working on stuff for Trend, but you're interested in, in, on your own and professional development.
[00:06:18] You come across, you watch a lot of talks, you're active on social. Everyone can see, actually I'll fire that up right now. You can see Jeff's Twitter handle on the stream. Um, you're active up on social, and, you know, you share a lot of great stuff that you've found, um, from other people in the community who are giving these talks.
[00:06:33] What kind of mistakes do you see people making with containers?
[00:06:37] Geoff: Um, I, I think the biggest mistake that I see people run into is, um, they're, they haven't made that switch from, uh, from being VM people, right? And I'm sorry, that sounds really mean, um,
[00:06:49] Mark: The VM people. Oh, no, no. But like people who are predominantly thinking of their software in terms of this is a VM.
[00:06:56] This is a VM.
[00:06:58] Geoff: So, so people [00:07:00] that are sort of taking early on in that, in that transition period where they're saying, okay, I've got this application that worked really well in this, uh, In this virtual machine that I used to deliver, um, and so they try to take that whole virtual machine and that whole giant application and, and bundle it up into this terabyte sized container image and try to run that in a, um, in the world, right?
[00:07:24] And, and I said terabyte and I didn't actually misspeak. I've actually had a customer come to me and say, we want to look at images that are a terabyte in size and I just sort of, I paused. And And I wasn't really sure how to respond to that because I think they were kind of missing the point. Terrible.
[00:07:41] It's big for even a VM though. Like, that's just big. I didn't want to ask what was in it. I just didn't want to, you know, I look at containers as, you know, um, you can build containers with an OS image or you can build containers just with your application in them, which is kind of what I prefer to do. [00:08:00] Um, and it's kind of the difference between, you know, a 10 megabyte image and a hundred megabyte image.
[00:08:06] Right. And it's like, if you can do 10, Why don't you do that? Because your attack surface is smaller, your, um, your downloads are faster, your startup is faster, there's a lot less to worry about.
[00:08:17] Mark: Yeah, and just, I mean, you know, if you get into terabyte images, how are you going to keep up fast changes in production, right?
[00:08:23] Even with gigabyte images, it's, it's, that takes bandwidth, right? That takes time.
[00:08:27] Geoff: Yeah, yeah, yeah, like, If you have a 10 meg image, you can download that in a very small amount of time, and you can get it up and running, right? Um, you can, you can, if you have a 100 megabyte image, or a gigabyte image, that takes a long time, it's not going to start up fast, you're going to have a hard time moving around.
[00:08:44] Mark: Mm hmm. Mm hmm. So, one of the things you mentioned there kind of implies something else. So, what do you feel is sort of like a point? Good, if I'm sitting and looking at my software and trying to decompose it into different pieces What is sort of the logical boundary for a [00:09:00] container because the technology will let me do whatever I want If I want to shoot myself in the foot with a terabyte image I can But how do I get down to those 10 meg images?
[00:09:08] Like how do I get down to that lean and mean one?
[00:09:11] Geoff: Yeah, so, um, the buzzword that, uh, that a lot of people use is microservices, right? Um, and I think the idea there is that you take, um, and I think that the word that people use in, in sort of the software architecture world is bounded contexts, right? And that might be a strange and weird word for people, but, um, kind of the idea behind it is you want your container, you want your service to do one thing, right?
[00:09:36] Um, if it's authorization, it does authorization. If it's Um, editing your profile. That's all it does, right? And so if you can break these things down, um, into your, your container image, is basically a simple service, it does a very small amount of things, or ideally one thing,
[00:09:54] Mark: then,
[00:09:55] Geoff: um, then you're good to go.
[00:09:57] Mark: Okay. Um, just a quick pause for everybody [00:10:00] on the live stream. You know, we see you from Mauritius, India, um, you know, uh, Belgium, a whole bunch of people kicking in from around the world, Argentina, India. If you have questions, um, please fire them off, uh, hashtags less talk cloud, um, hit them here in the LinkedIn live comments on Twitter, on YouTube.
[00:10:15] Uh, we'll address them in the conversation as Jeff and I go. Um, as you can see, we can go a little more technical than in the past couple of weeks, uh, which is great. I think that's really important because there is a lot of misconceptions around containers. So one of the things, you know, I mentioned earlier, I'm just really active on social and shares a lot of great stuff.
[00:10:33] One of the things you shared of the other day, Jeff, and no, this isn't like I dug through your Twitter for a gotcha, but was a paper that Google had written on container security. What were your thoughts on that? Can you maybe give us a quick little overview on what they did and then what your thoughts were?
[00:10:49] Geoff: Um, okay. Well, you did catch me a bit because I wasn't prepared for that question, but, um, I remember, I remember reading it and, and thinking, What a great thing for Google to post out [00:11:00] just, uh, you know, days before I'm supposed to come and, and, and be an expert on containers. Um, really, I, you know, go back and, and read the paper, uh, anybody who's watching.
[00:11:09] Um, it's, it's really an awesome paper because it, it does get into that, um, So, it starts you from the beginning of what's a container for, um, you know, being a packaging construct and getting stuff out there and then, um, starting to talk about different ways to secure your containers and, uh, and securing them at runtime as well.
[00:11:30] So, it, it kind of covers the whole gamut of, of sort of why containers exist, um, how to build them well. And then how to secure them, and it's a really great paper, and like I said, kind of made me like, Oh my, why am I going on this, instead of this person from Google?
[00:11:47] Mark: Fair, fair, and didn't, I didn't want to get you with a gotcha.
[00:11:51] Um, one of the things that, um, so I read that through after you sent it out, because I hadn't seen it, and then when I saw you tweet about it, I was like, okay, I gotta, I gotta dive into this. And it was [00:12:00] really, I found it was really well written from a, it was technical, but not technical.
[00:12:04] Geoff: Yeah, yeah, I found it was
[00:12:05] Mark: really approachable.
[00:12:06] Yeah.
[00:12:07] Geoff: You don't have to be an expert to, to really dive in and understand what's going on and really get it. Um, I really like the, the language that they use too. It's very, very easy to read.
[00:12:16] Mark: Yeah, and I dropped a link to that paper in the, in the LinkedIn chat for those of you on the stream. Um, so you can check that out and we'll tweet it out later.
[00:12:23] Um, but one of the things that came up in that, um, that I think you should be okay about talking about. Um, it was the idea of sort of supply chain. When it comes to containers was that, you know, in production there, they should be immutable. Um, and then there's all this stuff that happens before, um, you, you, you actually hit production.
[00:12:41] And then that needs to be part of your idea of not just the development and operations, but of security as well. Um, what, any thoughts on, on that sort of concept? Because we don't normally, we didn't used to talk about developing software in that concept. It was just kind of like, no, it's in production, you're good.
[00:12:56] Geoff: Yeah, um, so there are a few things in there. Let me, [00:13:00] let me try unpack a bit. I'm going to start with the supply chain thing, um, because I think a lot of current software development is, um, you kind of look for, um, you kind of look for a dependency, you look for a library that does what you're trying to do, you grab it, you ship it, right, and you don't spend, a lot of people don't spend a lot of time thinking about where it came from, um, sometimes they'll think about what the license is, but they won't necessarily look at how stable the, uh, how stable the project is, right, so you might find something that does what you want, um, Um, and then, um, you know, you're not really paying attention, and hey, it wasn't updated, uh, except for three years ago, right?
[00:13:39] And then, um, you've seen things where, in the recent past, where, uh, projects kind of die, and then they get taken over, right? Somebody says, hey, I want to take over that, that, uh, project for you. Um, and the person says, yeah, sure, I don't, I don't care about it anymore. Um, but the, the new owner, um, Uh, starts putting in new functionality that maybe you didn't [00:14:00] want.
[00:14:00] Um, like coin miners. Um, which, which has actually happened. Yeah, so, you know, sort of thinking about that supply chain and, and monitoring what's actually going into your container. Um, just like you would for, for any software that you're building. Um, you're, you're kind of worried about,
[00:14:18] Both: I don't
[00:14:18] Geoff: know if worried is the right word.
[00:14:19] You kind of have to think about, um, what you're bringing in. Right. In the same way that you would for any other software product.
[00:14:27] Mark: Okay. Um, yeah. And I mean, that's a new thing right now, especially so I'm torn. I want to circle back to something, but I also want to, that's a great segue into something else.
[00:14:37] Um, I'll circle back in a sec. Let's segue there. So you mentioned, you know, like if someone new could take it over and put something either outright malicious, like a coin miner, which we've seen a huge uptake in, in general, um, but specifically a few instances where it's happening containers. Um, but also that kind of brings in the idea of your container doesn't just exist.
[00:14:56] Like you don't just create it from scratch necessarily. A lot of the time you're pulling in other [00:15:00] containers as the baseline layers and you're pulling those in from registries. Um, now how does that, can you explain that flow a little bit and how that works and then sort of what you need to be aware of as a, as a developer?
[00:15:12] Geoff: Sure. Um, so for people who may not be familiar, um, when you build a container image, uh, it's basically, uh, a tar file. Um, hopefully people can guess what a tar file is. Uh, so it's a, it's a tar file. It's, it's got all the stuff that's in your, uh, in your application. But as, as you said, it's, um, it's kind of based in many cases, it's based on a lower level layer or lower level image.
[00:15:37] That's got, um, other stuff that you've brought in. Um, and then when you, when you bring all these things together, Um, you've got this file or a series of layers, right? Which is a series of files, and they get stored in a registry. And the registry is really just a web server. Um, that's, it's a file server that's storing, storing the files.
[00:15:58] Um, and you can either have [00:16:00] them in a cloud provider or you can run your own or you can use Docker Hub or any other, um, any other registry provider.
[00:16:09] Mark: Okay. Um, so that brings up a different one. So Docker Hub, um, from Docker, um, you know, Docker's undergone some significant changes in the last couple weeks.
[00:16:20] Um, right. And I mean, we don't want to dive into that. There's, there's obviously complexities and challenges there. But from a developer, if you're building stuff that's pulling down from a resource that's out there What, like, are you concerned about that? Is this, you know, so, I mean, you mentioned tar files, which are basically just one, you know, it's a zip file for, uh, you know, compressed bundle of other stuff.
[00:16:43] Um, and if you're pulling those in from other places, uh, you know, you just take them on faith or are you looking for issues? Are you reputation?
[00:16:52] Geoff: There's two or three things, there's two or three things there to think about. So the first one is, uh, is for like, does the, does the thing you're pulling in [00:17:00] do what you want it to?
[00:17:01] Um, and, and to steal a, uh, steal a phrase from somebody I know, uh, does it do what it want, what you're supposed to and only what Supposed to
[00:17:09] Both: Yay .
[00:17:11] Geoff: I'm not sure where I heard that before. Maybe it was you. Yeah. . Um, so you wanna make sure that, that your, your layers that you're bringing in, or even the services that you're bringing in are doing only what they're supposed to do.
[00:17:21] Mm-Hmm. . Um, but you also, one of the things that, uh, that you run into that you don't really think about until it happens. Is availability, right? So you're saying, Hey, I'm pulling in this image. Um, is Docker hub there? Yeah. And you know, 99 point some huge percentage, a lot of the time it's there. But actually, in in the time that I've been working with them a couple times, we've tried to get get through a build, try to pull down an image and Docker Hub was down.
[00:17:52] So you just you just lose that. And you have the same thing anytime you have an external dependency, right? So, you know, Um, you're, you're trying [00:18:00] to download the software from somewhere. Um, you kind of need to make sure that you've got a, a layer in the middle that you can trust and sort of have your own, under your own control that says, okay, I've got these vetted images, um, that I've, that I've taken from outside.
[00:18:14] I've kind of taken a look at them, made sure that they do what they're supposed to. And then they're also in your control in terms of availability as well. Right, so, um, if somebody says, oh, I'm going to delete this image. You're not out of, you know,
[00:18:28] Mark: out there
[00:18:29] Geoff: crazy.
[00:18:29] Mark: Yeah, yeah, yeah, you can say that, don't worry.
[00:18:32] Um, the, uh, so is that where, that's where the idea, that concept of like a private registry comes into play?
[00:18:37] Geoff: Yeah, exactly. Right, so you either have a private registry, Um, or yeah, some sort of proxy that's that's sort of keeping those those images for it.
[00:18:45] Mark: Yeah, makes sense, right? Because I mean, we've seen that not specifically in containers, but we saw that in the node JS world a couple years back when, for some odd reason, people use an external library to left pad strings
[00:18:57] Both: and
[00:18:58] Mark: someone yanked it right, like [00:19:00] entirely huge mass, like 200 some packages from one person got pulled and all of a sudden builds failed left, right and center.
[00:19:07] Both: And
[00:19:08] Mark: that's the last thing you want to do if you're trying to build sustainable software, is be dependent on something you have no control over.
[00:19:13] Geoff: Yeah, exactly. So, a lot of people, I think, sort of start in that, Hey, I'm just going to pull it from DrHub as part of my build, and then they go along with that for quite a while.
[00:19:23] And then, um, and then it's like, Oh, wait, I've got a dependency here that I didn't really think about. Um, and then, yeah, you kind of need to be careful with that.
[00:19:34] Mark: And sadly that, oh wait, is normally after something hits the fan. Yeah, it could be on a Friday afternoon when you're trying to do a release, for example.
[00:19:42] It's always on a Friday afternoon. The computer knows. They always know. Friday afternoon, likelihood of things exploding through the roof.
[00:19:50] Geoff: Yep.
[00:19:51] Mark: Um, so one of the things I wanted to circle back to, and we had this conversation, uh, after the disastrous start this morning, Um, that I, I had mentioned to you on the, um, so [00:20:00] Disney Plus launched last week, Um, and I don't have any knowledge of the internal workings, Um, but it seems like they've taken a microservices design and we touched on that a little bit earlier and you kind of use It a bit in your example, but for me, so I signed up.
[00:20:12] I've got kids I was excited to watch the Star Wars stuff on my own and the Marvel stuff and I signed up and I thankfully it Works a second try for me to sign up But then it had intermittent errors throughout the, the, the week. And basically, for those of you that hasn't signed up for Disney Plus and know, this isn't an ad for it.
[00:20:27] Um, but there's five main content pillars. There's the Disney, the Pixar, the Marvel, the Star Wars, and National Geographic. Um, and once you've signed up, each of those throughout the week were, uh, Down at different times for me, so I couldn't get to like the Marvel stuff or the Pixar stuff or National Geographic But the others were up and running and or I couldn't edit my profile pic And but I could still watch things right or if I'd started it I could watch it and for me, you know being nerd I was like, wait a minute.
[00:20:53] This feels like they're all separate services that are having some problems But the others [00:21:00] aren't. So as a user, instead of just getting, you know, the annoyingly cute error message with Vanellope and Wreck It Ralph saying, We can't do this for you. Um, which is great the first time you see it, but after like the sixth, you're like, I get it.
[00:21:12] I get it, Vanellope. I can't get my data. Um, but it was a better experience because at least I got something. Now that for me kind of screamed microservices. Yeah, and as we've had a previous conversations microservices are kind of like linked hand in hand with containers
[00:21:28] Both: Yeah,
[00:21:29] Mark: but they're different right a microservices is an architecture design containers a way of delivering something Why are they together so much?
[00:21:37] Geoff: Um, I think because when you have Microservices you have a whole bunch of stuff that you want to deliver, right? So you've got instead of this one monolithic Application, right? Um, that, that's just sort of, it does everything and it goes and it all crashes at once and you get the, uh, you get the whale on the [00:22:00] screen and nothing works, right?
[00:22:02] Um, instead of that, you've got all these different services and you get sort of progressive degradation, right? And so you get little things that break here and there. And because containers are really the, the technology around containers and deploying containers and orchestrating containers has progressed.
[00:22:21] So that you can deliver a lot of stuff really easily, right? And you can tie them together really easily, right? So, um, it gives you that ability to architect your application in those microservice, in that sort of microservice fashion where you have, okay, I've got this small service that can grow or shrink or fail independently of everything else.
[00:22:44] And you've got this on. So that's great architectural development, but then on your on your deployment perspective, you can still deploy that thing right really easily because you've got this container that you can just shove somewhere and just run somewhere and having that environment that [00:23:00] you can just make these little services deploy rather than having to have all that I mentioned at the beginning about having a whole VM.
[00:23:10] You have to worry about it and all the stuff that, uh, that, um, that can go wrong when you're trying to deploy one service onto a VM, right? Instead of having one service, you have a thousand services. And you imagine having a thousand different VMs that all have to be configured right. Um, and it's just, it's just a disaster, right?
[00:23:29] So, I think the containers really help to enable. That, uh, that microservice approach because you can easily package up everything that you need to and ship it out and run it and have it, and have that sort of uniform interface where the, uh, the application just comes up and presents a network interface and you're, and you're good to go.
[00:23:49] Okay.
[00:23:50] Mark: Um, so that, we're dancing around a big K word and we're going to keep dancing around it for a second. But, um, that actually leads to something. So, so for you, your role at Trend, you [00:24:00] work on our container image security product. Right. Uh, it was previously smart check. Now it's, uh, under the cloud, the new cloud one, uh, naming, uh, which, um, itself actually deploys as a set of containers, right?
[00:24:15] Yeah. Cause. So that's a bit different, right? It's only up in the last year or two that we've seen people actually developing software and deploying it out as a container, um, or series of containers. What sort of challenges have you seen there? Because, I mean, you've, you've packaged software up the old way, you've worked on, you know, global scale cloud services, what, what is now, what's different, what's better, what's worse?
[00:24:39] Geoff: So I think this is where the keyword comes up, right? So we started talking about Kubernetes, and that's how the application that I work on is deployed, right? You need to have a Kubernetes cluster to deploy it. And the nice thing about that is that, um, Kubernetes is a really great [00:25:00] place. It gives you a lot of capabilities built in, right?
[00:25:03] In terms of, like I was saying earlier about that application orchestration, right? It's like I can say, I want these six services to come up and run, and they all need to be able to talk to each other. And it's really easy to define that. And, you know, when they fail or crash, or, well, they never crash, of course.
[00:25:22] It happens. When they do crash, because of whatever happening, then the infrastructure will bring them up and start running them for you. That's super amazingly awesome, right? Before, I would have had to write that all myself, right? I would have had to, you know, take the, uh, sort of replicate the combined intelligence of everybody at Google who worked on the Borg system before it became Kubernetes, but it's fun.
[00:25:50] For an inspired Kubernetes, um, all the thousands of contributors that, that make Kubernetes better every day,
[00:25:57] Both: right?
[00:25:57] Geoff: Um, I would have to do that with my [00:26:00] team. Um, and that's just insane, right? Um, trying, trying to build something that's, that's as big and complicated and useful and fascinating also.
[00:26:11] Mark: And none of that's unique to the problem you're tackling either, right?
[00:26:13] So why solve it in one place to have to solve it in a bunch of others?
[00:26:17] Geoff: Yeah. And, and I'm, I'm, I'm not gonna attribute this, this quote that I'm gonna steal from, from someone famous. Um, but it's, it's undifferentiated heavy lifting, right? And I think that's, uh, I think a lot of people were, will recognize that we didn't want to do any undifferentiated heavy lifting, right?
[00:26:33] We said we want to focus on our application and get it out there and get it running and not worry about a lot of those, those complex details that really don't help us. Deliver business value to our customers.
[00:26:47] Mark: So question for you then. So comparing that, you know, everything said totally makes sense Especially, you know leveraging the the power that's already built into kubernetes If you have to ship a patch Right and like a traditional what people would [00:27:00] think of as a traditional patch Are you then just sending out a new version of the affected containers like saying?
[00:27:05] Hey, you can pull a new version Like how does that how does that flow because that's got to be smoother
[00:27:10] Geoff: It is. It's super smooth. Um, so the way it works is, uh, we basically just do a whole build of the, the entire service, ship out a new version of the container and, and it's done. Right. Um, and then our customer will just sort of update their lineup and it's sort of like a one line command and they can pick up the new, uh, the new container and it just gets replaced and there's, there's nothing to worry about.
[00:27:35] Right. Um, it says it does require you to. Build your software in a way that it can be updated that way, right? There are some things you shouldn't do when you're building applications that need to be upgraded that way. But, uh, but it's really makes it super easy,
[00:27:53] Mark: right? That, that seems far smoother, like even, even taking those architectural design sort of choices on your side [00:28:00] as the, as the software builder into play, that seems way easier than our old method of like, here's, here's a patch, Now, you may or may not have to take the system offline to upgrade this patch.
[00:28:09] It may have additional dependencies because I'm not sure what it's running on top of. Um, you're, you're reducing the order of the complexities by an order of magnitude or more.
[00:28:18] Geoff: Yeah, um, and actually I'm going to change the word for you. Um, I'm not reducing it, I'm moving it to someone else's problem. Even better.
[00:28:26] Yeah, exactly. Like, because, because the, the container environment is, is predictable. Right. It's like, and again, I'm going to go back to that most of the time. Um, so there, there's a great podcast that, uh, that I tweeted out the other day, um, where they interviewed about all the differences in the underlying, um, uh, the underlying container environments.
[00:28:49] It's like, Yeah, I'm glad that most of the time I don't have to think about that. Because most of the time your containers just run, it looks like a Linux process that's just [00:29:00] going. And you don't have to worry about it.
[00:29:02] Mark: Well, that's, that's the advantage, right? I mean, you hit on a couple words besides the undifferentiated heavy lifting, um, you know, what are you doing if it's not delivering value?
[00:29:09] Like, why are you working on that stuff? And the more you can kind of push away, the better, right? I mean, that's the whole advantage of cloud is going up that shared responsibility model to the right as far as you can to get away from doing stuff that doesn't make a difference, right? Um, so question for you then, uh, with Kubernetes, like staying on Kubernetes is a second here.
[00:29:32] Um, what do you think? So, you know, it's all the rage. Everyone's like, Oh, we have containers. We need Kubernetes. It's, you know, they're so intrinsically linked now. Um, what do you think? So I think we know what the biggest plus is already. It's getting rid of that undifferentiated, heavy lifting, taking advantage of all that, um, complexity and letting someone else handle it.
[00:29:51] Um, if it can't be eliminated, what's the biggest downside of Kubernetes?
[00:29:56] Geoff: The biggest downside is people don't think about all the complexity [00:30:00] that is still there, right? Um, because just because it's hidden behind this, this magical Kubernetes thing, doesn't mean it's gone anywhere, right? Yeah. And so if you're trying to, if you're trying to run it yourself, if you're trying to build your own cluster, if you're trying to manage your own cluster, it's a surprising amount of work, right?
[00:30:23] To get everything up and running and Um, you know, like, sorry, up and running is kind of okay. Keeping it running, keeping it alive, keeping it
[00:30:34] Mark: Two very different things. Up and running, I have a container I deployed. Keeping that going no matter what happens. Very different proposition.
[00:30:43] Geoff: Yeah, exactly. So, there's quite a lot of expertise involved in in having a Kubernetes system that works properly.
[00:30:50] Both: Um,
[00:30:51] Geoff: so a lot of people think that they can just say up and running. Follow a tutorial or spin up kubernetes in their [00:31:00] in their data center and and have it work really well. Um, and unfortunately, a lot of times it doesn't work very well for them because it is such a complex task. Um, luckily, a lot of cloud providers will give that to you that like a lot of them have a really good kubernetes setups where you can say, push a button and you've got you've got a cluster.
[00:31:22] And those tend to work really well, like our customers have done really well with that. The ones that are sort of spinning up their own clusters, it tends to take them a while to develop the expertise and develop the understanding of all those little things that can go wrong with networking or storage or, you know, You know, the things that, that it doesn't really paper over very well, um, and you kind of need to have a lot of expertise in.
[00:31:48] Mark: Yeah, because I mean, it's, it's in itself, it's now becomes critical infrastructure, and it's another layer of abstraction, and you know, when, anytime you're putting any layer of abstraction in, you need to make sure you're getting more value versus [00:32:00] the effort. Right? And that's why, you know, I know I still talk to a lot of, uh, infrastructure teams who are excited about it because it's something where they can, they're still, if you need to take advantage of assets you already have in your data center, it's a great way, uh, to do that.
[00:32:16] But more and more I'm seeing the same thing you are, where people are taking advantage of the, um, you know, like the, uh, GKE on Google. E. K. S. On a W. S. Microsoft's kubernetes service more and more of like, Hey, I want all the good stuff, but I don't want to deal with all the mountain of stuff in the background.
[00:32:32] But I mean, that's the whole point of cloud, right? Is like, I don't want to do it. You can do it and I'll pay you a couple cents an hour to do it.
[00:32:38] Geoff: Yeah, managed services like they make a huge amount of sense.
[00:32:42] Mark: Yeah. Um, so adjacent topic, um, you know, and I mean, based on, on your history, uh, you know, you've worked in telecom, you've worked on, on massive scale services, you're deep in containers now, um, one of those sort of keys is, is obviously reliable, um, but sustainable software, um, [00:33:00] have, do you think containers have pushed that ahead?
[00:33:02] Is it easier to build sustainable software using containers? Um, and sort of just what's your general take on how to approach it?
[00:33:09] Geoff: Um, I think that more, more than containers themselves, like if you, yeah, actually, yes, I think the short answer is yes. Um, I think because it does enable you, it does make it easier for you to say, here is the whole package,
[00:33:27] Both: right?
[00:33:28] Geoff: Um, here is, uh, you know, to jump into the technical term, here is the docker file, right, that defines this application. Here is where you get every single piece of, uh, of infrastructure that you need to build this application and, and run it. Um, and like compared to in the past where you said, Oh, I need to have rel seven.
[00:33:51] Um, so sort of you have that line item somewhere off in a document somewhere. And you have this, this version of Red Hat Enterprise Linux and [00:34:00] these libraries that come from where again? And then, um, and then you, and then you install your application, right? Whereas if, uh, with the Dockerfile sort of concept, it's like, Hey, I've got this base image, which of course I've kept a copy of in my private registry for utility purposes.
[00:34:18] Um, and then, and then you have the exact instructions for building that application from scratch. Right. And, uh, and that I think makes it a lot easier to sort of maintain that over time, right? And even if you don't, uh, even if you don't touch a service for a couple of years, you can go back to it It's like, hey, it's still built, right?
[00:34:38] Because you've still got that exact environment set up.
[00:34:41] Mark: You've essentially got that sort of single source of truth. You know that file. It's not like a document that we hope you updated, which nobody ever does. You've got literally, this is the executed code that creates that thing. Exactly. Nice.
[00:34:55] Geoff: That, that helps a lot.
[00:34:56] Mark: Okay. We've hit that time in the stream, so, uh, you know, everyone's [00:35:00] still tuned in, it's fantastic, we still got people around the world, um, I know you've watched the first two that we did of these, uh, it's always a ton of fun, I'm gonna hit you with a bunch of rapid fire questions. Alright. And I want short, like, one or two word answers and then we'll circle back if anything crazy comes up.
[00:35:13] Alright. Um, and I always find it interesting, cause I've, I've known you for a while and you're such a good friend, I think I know how you're gonna answer these. But I'm curious to see how well I know. Maybe we'll evolve this to the point where it's like a game show. But right now we're going to just keep it straightforward with a rapid fire.
[00:35:29] Alright. So, Unix philosophy. Yes or no for containers?
[00:35:33] Geoff: Yes.
[00:35:34] Mark: Okay. Databases. Containerize them or no? No. A giggle and a no. Okay, we're coming back to that. Uh, blue green deployments. Real thing or just lip service?
[00:35:47] Geoff: Um, I'm only allowed two words. Uh, real thing.
[00:35:51] Mark: Real thing, okay. We'll come back. Uh, can containers be serverless, yes or no?
[00:35:56] Geoff: Gah. [00:36:00] That is my word. That's your, that's your answer?
[00:36:03] Mark: Your answer is gah? Okay, fair. That's a legit answer. Um, Kubernetes. Most hype ever or just the most hype of 2019?
[00:36:13] Geoff: Um, I heard more ever.
[00:36:16] Mark: Okay. Uh, so, uh, containers legitimate solution that you should be aiming for, or just a way station to the real one That's serverless.
[00:36:27] Oh.
[00:36:28] Geoff: Oh, I'm gonna get in trouble for this one. Yes, you're . Oh. Um, Ben, if you're watching this, I'm sorry. Containers are real. They're, they're a good thing. Um, I keep going. I keep going. Okay.
[00:36:44] Mark: Okay. Um, we will loop back in Ben Kehoe and see where, where that is. Um, okay. Last one. Languages. Go is better than Java in 99 percent of cases.
[00:36:53] Yes or no?
[00:36:55] Geoff: Ooh. I'm only allowed two words? Today, I'll say yes. [00:37:00]
[00:37:00] Mark: Okay. Interesting. Very interesting. Uh, you held yourself well. Uh, so, um, you know, UX philosophy was straightforward. You were a firm, rapid no on databases and containers. Why?
[00:37:15] Geoff: Um, there's, there's a lot smarter people than me who have talked about this.
[00:37:21] Um, but it's effectively, um, Containers are really good at starting and stopping and getting rescheduled to different places and, um, that doesn't work really well with databases, right? Databases like to be on the same system as their storage. Um, they, they like to have storage really closely attached to them, right?
[00:37:41] Um, and so if you have a container that's running a database and it dies or gets rescheduled or the node that it's on gets moved, Um, and it hasn't done the right things to its data storage, then bad things happen. Um, and so, yeah, don't run your database in containers [00:38:00] unless you absolutely have no choice.
[00:38:01] And then still don't. Um, find a managed database service that you can connect to.
[00:38:07] Mark: Yeah, and I think that's a good point, and I mean, I 100 percent agree. Um, and I think not that I have to, but I do. Um, but I think it's important to realize that containers don't solve every problem, right? So there are applications and use cases like databases where you're in for a world of hurt if you try to go down that path.
[00:38:24] Geoff: Yeah,
[00:38:25] Mark: right. Um, so blue green deployments, you were a little iffy there. Uh,
[00:38:32] Geoff: so there are, there are some really awesome mature teams out there. And I think. There's a lot of teams that are getting that, right, those are blue green. Um, I think that it's a maturing technology. It's not something that people know how to do out of school.
[00:38:50] Right, and I could be wrong, it's been a long time since I was in school, but I'm not running into people that sort of know natively how to do this. And yet, [00:39:00] I talk to a lot of really smart people that are doing it for real. Alright, so
[00:39:05] Mark: is it fair to say it's just like the concept is solid, it's where we should be going, it's just a matter of comfort level.
[00:39:11] You know, especially people moving out of dev more into ops, like it's a really tricky shell game ops move, right?
[00:39:18] Geoff: Yeah, I think that people need to understand how to do it. Um, but once they do, it's like, oh, this is amazing. Yeah. It's so helpful.
[00:39:26] Mark: Yeah. And it's super easy to draw on a slide . Yeah. It's a whole nother thing to actually do.
[00:39:32] Right? Uh, last one I wanna circle back to, uh, containers can be serverless, yes or no. Um, and you know, you literally apologize to, uh, Ben, for those of you who don't know, Ben Keho, an AWS uh, serverless hero. Uh, works at iRobot. Gives a ton of phenomenal, uh, technical talks. Huge leader in the serverless community.
[00:39:51] Phenomenal guy too. Um, but okay, why, what was your stance there? Like, why do you think that's so, uh, controversial as far as [00:40:00] containers can be serverless?
[00:40:02] Geoff: Um, so the reason that, that I, I really hesitate with that is because I think that containers are, you know, they're really not the, where, where you want to think about.
[00:40:17] When you're talking with serverless, right? Yeah. This is a lot about, um, using managed services for as much as you can,
[00:40:26] Both: right?
[00:40:27] Geoff: It's about not thinking about or not needing to think about where your software is deployed. And containers kind of get you part of the way there. Um, but I think you to really get to sort of the serverless philosophy.
[00:40:43] You really need to scale back even from your, your sort of microservice thoughts that are going into your container and saying, okay, I just want my code to run somewhere and I don't want to be worried about how I'm building that, right? And so you say, If you're building a container, I think there's people who are, [00:41:00] who would, who would argue that, that if you're building a container, you're doing too much work already, right?
[00:41:05] Fair.
[00:41:07] Mark: Yeah, and I mean, this kind of combines with the other rapid fire I asked you about as far as containers being a waypoint or, you know, are they an actual destination? Um, and, and let me sum it up, because I think, uh, reading between the lines, I, I think you'd go, and I know how I feel on it. Whereas, depending on the problem you're trying to solve, Ideally try to do something serverless because that's the least amount of work you need to do.
[00:41:29] Geoff: Absolutely.
[00:41:30] Mark: Then go containers.
[00:41:32] Geoff: Yep.
[00:41:32] Mark: Then go VMs in a sort of falling filter scale of if you can't solve, if it doesn't fit the right way. Cause don't shoehorn it into any one of those. Find the one where it slides in like a glove.
[00:41:44] Geoff: Exactly. You gotta find the right tool for the job that you're looking for.
[00:41:47] Um, if I can sort of plug for a little bit. The right tool in a lot of cases, in a lot more cases than you might think about, is serverless. Um, sorry, container people. Um, [00:42:00] because, you know, you take your function, you deploy it to the world, and it just runs. And you don't have to worry about a lot. If you're not ready for that yet, then containers are a really great solution, right?
[00:42:12] So a lot of the stuff that I build necessarily is in containers because that's kind of, it's what works for deploying software a lot these days. It's a lot harder to deploy. Uh, serverless applications still, if you're talking about software, if you're not talking about a service that you're running in house, if you're talking about delivering software to people, then containers are still really, I think they're really the right place to go.
[00:42:37] Mark: Yeah, and I mean we had this this chat earlier offline when I was telling you I had serverless conf I was talking to Brett McGowan who's a developer advocate at Google around Google Cloud Run, which is a somewhat controversial container Slash serverless function service, but the way Brett Prevented it was really really when Brett was talking about it was really really logical to me was And it reminded me it [00:43:00] was a lot of the time when we're deep into this tech You start to think either or Like, it's this, or it's that, or it's this, and you know, if you're building new, you've got all the choices in the world.
[00:43:09] But the vast majority of work that's being done out there is on stuff that exists today, that you can't justify rewriting completely from the ground up in your ideal framework, or model, or whatever. You've gotta figure out how to optimize what you've got. While moving forward. Okay. Fantastic. I mean, I agree it's in the ideal world.
[00:43:31] We wouldn't have to build any of this, right? You'd be able to just to go back to like the old, almost visual basic level days where I draw a button, I click on it and tell it what to do when it's clicked. And I don't worry about anything else.
[00:43:42] Geoff: Yeah.
[00:43:42] Mark: Right? Like, I mean, that's the, you want that direct line of value.
[00:43:45] So, um, we're almost at the end here. This has been a fantastic conversation, Jeff. Uh, a couple quick questions, or relatively quick. What's the weirdest problem you've ever seen dealing around containers or even something non container? Just weird sort of tech [00:44:00] problem tackling these kind of things.
[00:44:02] Geoff: Uh, let's see.
[00:44:04] So I think the weirdest thing that I've seen is that that customer with the terabyte image. Um, the weirdest tech problem I've tried to solve. Um, I think, I think that a lot of the stuff that I'm running into lately has been around, you know, just the, the fractal nature of Kubernetes, right? It's like, and digging into, um, why something isn't working today.
[00:44:34] Sort of the networking stuff that you kind of find in there, you know, and, and I've, I've done like actual Linux kernel level work, understanding the kernel and how, how packet work, how packet forwarding works there and still, um, you kind of get lost with like, what's going on here?
[00:44:54] Mark: You forgot your quotes around how it works.
[00:44:57] Oh, yeah. You're like, it goes in [00:45:00] this function and then it's delivered somewhere.
[00:45:02] Geoff: Yes. Yeah, so I used to know now it just it just doesn't make sense anymore.
[00:45:08] Mark: Sometimes learning more is not the right answer Sometimes just accepting that you know what it just did a weird thing. That's okay. That's fair.
[00:45:16] Geoff: Yeah.
[00:45:18] Mark: All right So a more personal one here. What's your favorite cloud service doesn't have to do anything with do with containers Just using you know either in Google in Azure or in AWS That's something that you find useful for your own stuff, whether it's work or personal learning or anything like that.
[00:45:32] What's your favorite cloud service?
[00:45:34] Geoff: I'm going to say Lambda. Okay. So, yeah, um, there's others that I want to play with more, right? Like I played with Cognito a bit, so AWS Cognito, um, it's really awesome for doing, sort of user management. That sort of thing, you get stuff really quickly. Um, but Lambda is, um, is the awesome.
[00:45:58] Um, you just, [00:46:00] you, you plug in a couple lines of code and magically attach it to some events and the right things start happening. Right, so just having that, that ability to do stuff like that just blows my mind.
[00:46:13] Mark: It is pretty amazing. And you know, we just, uh, this week or last week was the five year anniversary?
[00:46:18] Yeah. Of, uh, of Lambda? Yeah. Which is nuts that we've come that far where when it first announced everyone's like, hey, what's this good for? Um, and then very quickly to go. Oh, yeah. No, we built the entire solution on lambda and it can you know It's not just glue anymore. It's legit computing and you know reinvents 12 days away And it's almost guaranteed that we will see more announcements around Lambda and hopefully a ton of new container announcements as well
[00:46:46] Mark: Right on. Jeff, thank you very, very much. This has been fantastic. Great conversation. Thank you to the audience for joining in from around the world and for your patience this morning. I think we rebounded strong, which is good. We've had folks, [00:47:00] like I said, from all around the planet tuning in again, which is wonderful.
[00:47:03] Please keep those questions coming. Hashtag Let's Talk Cloud. Um, we will be back again next week with a fully working functioning stream covering another topic in cloud. And if you have topic suggestions, not just questions on the existing topics, let us know. We are happy to tackle whatever this really is.
[00:47:22] Community driven. This is what makes it interesting. Personally, I just love talking to smart people, but hopefully you guys are getting something out of this as well. So Jeff, thank you again. We very, very much appreciate it. Have a great rest of your afternoon, Jeff and everybody for tuning in. Catch you later.
[00:47:38] [00:48:00] Thanks.
