WEBVTT 00:00.000 --> 00:11.760 Thank you all for being with us. It sits 30 p.m. Actually sits 31 p.m. I'll make sure this 00:11.760 --> 00:17.360 taught us really small and quick so that we all can live and have our favorite refreshments 00:17.360 --> 00:23.760 and enjoy some delicious, built-in food. Again, thank you. So, what does top sprangler 00:24.080 --> 00:33.600 well, I'll come to that slide. Awesome. I hope you can hear me now. I'm not going to repeat what 00:33.600 --> 00:41.680 I said. Well, all right, the agendas slide is here who are we, I will just go through what 00:41.680 --> 00:48.240 who are we, what is that swanler, why are we doing this and give you a very brief overview of 00:48.240 --> 00:54.240 the system? Ethan will give a demo then it's all up to you to ask the questions and we can 00:54.240 --> 01:01.360 head up soon. So, I'm Carvee. This is Ethan. We come from Griker. It is for those who don't 01:01.360 --> 01:08.720 are Griker. It is one of the biggest insurance companies in U.S. It's highly regulated and we come 01:08.720 --> 01:15.600 from an operating systems, containers and runtime team and our mission is to deliver compliant 01:15.680 --> 01:22.000 and secure operating systems, containers and language one times. So, what is top sprangler? 01:22.000 --> 01:28.240 Well, I would like to call it the brothel that I'll download the image building and management 01:28.240 --> 01:38.480 where we get to build based images, container images and effective secure way where we can 01:38.480 --> 01:45.280 reduce the number of redundant processes and operations which are integrated fully with the 01:45.280 --> 01:54.320 CI pipelines and what did we build it? Well, there are several open source tools. One of the best 01:54.320 --> 02:00.960 is that they have, there are many of them where we can build a single golden image in a secure 02:00.960 --> 02:08.080 efficient way. What we wanted was like for our use cases, we have like multiple versions of 02:08.080 --> 02:14.080 Ubuntu, say like two versions of Ubuntu, four versions of Pareto and like three versions of 02:14.160 --> 02:21.440 Tomcat. Last thing we want to do is build them 24 times. So, one config file where we get to have 02:21.440 --> 02:29.600 all the versions specified and then build them without redundancies, manual interventions and 02:30.400 --> 02:36.880 the best part is we also have like a lot file through which we get to maintain the 02:36.960 --> 02:44.640 versions of the dependencies in a accurate way. It's actually what we want. Our design 02:44.640 --> 02:51.200 considerations where scalability, flexibility, making sure it is fully integrated with the CI 02:51.200 --> 02:56.080 pipelines and efficiency. That said, I'll give them my two items. 03:00.640 --> 03:06.160 All right, can everyone hear me all right? Awesome. So, over here we have a graphic showing the 03:06.240 --> 03:10.720 overall design and what we were looking for in an image building system. So, the most important 03:10.720 --> 03:16.640 part is that it's automated. We previously came from teams where automation wasn't a priority 03:16.640 --> 03:22.400 and that creates a lot of ops burden on the team itself. And so, this really influenced our 03:22.400 --> 03:28.640 design and we started with a nightly runner that updates our configuration. And so, we wanted to 03:28.640 --> 03:37.520 write our definitions of code as configuration. And so, by the design we decided we have a 03:37.520 --> 03:44.240 lock file that is machine generated. We have a configuration file that is human generated. And 03:44.240 --> 03:49.760 those two work together to create a history of all changes of versioning that happens. And that's 03:49.760 --> 03:55.840 great for auditing purposes. It also allows us to have closer to reproducible builds. We do do 03:56.160 --> 04:01.040 a little bit of updating so that it's not 100% reproducible right now. We are working towards 04:01.040 --> 04:06.880 that in the future. We also want to integrate automated testing and then have automated publishing 04:06.880 --> 04:14.080 all using GitHub actions with the touch-rangler tool itself. And so, one of the greatest parts 04:14.080 --> 04:20.160 of touch-rangler is automated dependency updates. And we start with Docker images. So, we need to 04:20.160 --> 04:26.240 find our base image. In most cases, it's a boon too. We also can support red hat. But any Docker 04:26.240 --> 04:31.360 image can be used as a base image in touch-rangler. We then specify the tags that we want to 04:31.360 --> 04:38.640 track in using mutable tagging like Jamie for a boon too. We can automatically pull the latest version. 04:38.640 --> 04:43.600 So, in this graph up here, we start with our configuration. In this case, a boon to Jamie. 04:43.600 --> 04:50.880 We then pull the image from Docker and fetch the actual versioning like 22.04 Jamie. We also 04:50.880 --> 04:55.520 track the registry and the digest of that exact image. And that's written to the lock file. So, 04:55.520 --> 05:01.120 this means every night when we update our lock file, we're seeing the exact digest that all of our 05:01.120 --> 05:07.200 images are being built with. We also do a similar process for a GitHub. For instance, Amazon 05:07.200 --> 05:12.960 Coreto, which we've decided to use as our Java runtime, is hosted on GitHub. And here we fetch 05:12.960 --> 05:17.600 the GitHub repo. And then we look through all of the branches. And we pattern match those with 05:17.600 --> 05:23.280 our target version. In this case, it was Amazon Coreto 21. So, after we fetch the versions, we 05:23.280 --> 05:30.400 match the target. And then we write up those exact versions to our lock file. So, I'm going to 05:30.400 --> 05:36.240 do two different demos. They're pre-recorded. The first one will show how we build Java runtime 05:36.240 --> 05:40.400 using Amazon Coreto with Touch Wrangler. And then in the second ones, a little more abstract 05:40.400 --> 05:46.960 example on the powerful capabilities of Touch Wrangler. And so, again, building simple runtime 05:46.960 --> 05:51.520 languages. And this case is going to be Amazon Coreto. So, we start by defining our base. Here, 05:51.520 --> 05:55.200 we're defining a boon to. We're telling it, which versions we care about. In this case, it's 05:55.200 --> 06:01.120 Jamie and Focal. We define the package manager. And then we have a couple extra fields that are 06:01.120 --> 06:06.000 used for the internals. And then the most important part is where we fetch the version. This 06:06.000 --> 06:09.760 tells us that we use Docker. It tells us how to find the image. And then there's a command that 06:09.760 --> 06:15.200 we run to fetch the actual version. This allows it to be extremely expandable for any different 06:15.200 --> 06:20.320 operating system or any different base that we would want. We can define our own command and fetch 06:20.320 --> 06:27.680 that version. So, next, we define a feature. In this case, Amazon Coreto. And we can specify the 06:27.680 --> 06:32.800 versions we want to look for. And that's used for the pattern matching on the branches or tags. 06:32.880 --> 06:39.760 In this case, we're tracking the major LTS versions. So, Amazon Coreto. 21, 17 and 11. And then 06:39.760 --> 06:44.640 we specify that. We fetch the version by using GitHub. And we just provide the organ the project. 06:44.640 --> 06:50.000 We do have a lot of support for templating in most fields, as you can see in version tag in project. 06:52.960 --> 07:00.000 And then we get to define the installation. And so, here we're setting the Java home. We use app 07:00.000 --> 07:06.880 to use Amazon's repositories to install the desired version of Java. As you can see here, 07:06.880 --> 07:11.520 we're using a lot of templating for the versioning. This allows us to write the versioning once. 07:11.520 --> 07:16.480 And no matter what version we want, we can get it. Once we go to the lock file, which we'll see 07:16.480 --> 07:26.400 in a minute, we'll see some more rigorous versioning. So, we can also define the installation for 07:26.400 --> 07:31.840 separate package managers. The previous example was using app. In this same configuration, 07:31.840 --> 07:36.560 we've also defined the installation for young. And so, this means that no matter what package 07:36.560 --> 07:41.520 manager we're using, if our base has apped or young, we can install Amazon Coreto on it. No 07:41.520 --> 07:48.240 matter what version of Amazon Coreto we want, or no matter what type of after young repositories 07:48.240 --> 07:53.600 we're using for installation. Once you do that, we generate our lock file. So, here we have the 07:54.160 --> 07:59.600 booty base from before with the exact versioning we're using as well as the digest. So, every 07:59.600 --> 08:04.080 image that's built with this lock file, we'll use this exact digest every single time. 08:06.000 --> 08:12.240 We also have a definition of Coreto here. In this case, it's version 11. The 11 in our target 08:12.240 --> 08:19.280 version was normalized to 11 0 2641. And as you can see on our installation, we're no longer 08:19.280 --> 08:24.480 insulating by template. We actually have the specific version in the install steps. 08:26.000 --> 08:30.400 So, here's a brief recording. I'll start by showing that we have our Wrangler.com. 08:30.400 --> 08:35.600 Well, that's the human written configuration file in the entry point. And this is just showing 08:35.600 --> 08:44.000 that it has the examples that we had before. Here we have Amazon Coreto 811 1721. And we have 08:44.720 --> 08:51.360 bases Ubuntu and REL. And so, we're starting by fetching the versioning from Docker. So, we're 08:51.360 --> 08:57.440 pulling the Ubuntu jammy image. We're going to run our versioning command on that. And then once 08:57.440 --> 09:01.920 that's done, it'll be written to the lock file. And we're also doing the same for Red Hat. Right 09:01.920 --> 09:07.040 now, things are working synchronously. We're working to make things happen more synchronously 09:07.040 --> 09:11.920 to speed up this process. But since it's normally done in CI, we don't typically care about 09:11.920 --> 09:17.040 how long that takes. And then we fetch all the versions from GitHub. And finally, we fetch our 09:17.040 --> 09:22.480 digest. And then we have our lock file. So, you can see the lock file Wrangler.lock. As well as Wrangler. 09:22.480 --> 09:27.520 Text which lists out all of the images that we're creating. So, with that single configuration 09:27.520 --> 09:32.880 file, we've defined all eight of these images. And I'll briefly show the lock file where you can 09:32.880 --> 09:37.520 see that we no longer have that loose versioning from before. Here we every version is specified 09:37.520 --> 09:46.880 completely. Instead of Credo version 8, it's 844206. We'll see another version of Java 21. And then 09:46.880 --> 09:52.560 another version of 17. All of our installation is extremely explicit. And this lock file defines 09:52.560 --> 09:59.920 exactly how we're building our images. So, the previous example showed how we can use 09:59.920 --> 10:06.320 correct or build a variety of Amazon Credo-based images with a single configuration file. But what if 10:06.320 --> 10:11.520 we had different types of runtime layers that we wanted? Suppose we wanted to create Java 10:11.520 --> 10:16.320 images, but we also wanted to create Python images as well. This next example is going to show how 10:16.320 --> 10:23.920 we can use Touch Wrangler to create all possible combinations of a version set. So, here we're 10:23.920 --> 10:29.680 defining a dependency A1. There's going to be two A dependency. So, there'll be A1 and A2. Here 10:29.680 --> 10:36.720 we're saying that we're just going to copy this file from my local system. A version with the 10:36.720 --> 10:41.440 version dot text. And we're going to copy that into the Docker container. And we specified versions 10:41.440 --> 10:48.160 1, 2, and 3. Then we have A2. And so, in this example, A1 and A2 are going to be conflicting 10:48.160 --> 10:53.920 features. So, we're going to say we want either 1 version of A1 or 1 version of A2. We never want 10:53.920 --> 10:59.280 both A1 and A2. Here we have versions alpha beta and gamma. And we're again copying those 10:59.280 --> 11:06.800 files to our computer. From our local system to the container. Then we're going to have two 11:06.800 --> 11:12.800 features called B or two feature B. And then in this case we have B1 versions 1, 2, and 3. And 11:12.800 --> 11:20.320 there we just echo the version into a file on the container. And B2 is similar. We have B2 with 11:20.320 --> 11:25.440 the version alpha beta gamma. And then we're just writing that version to a file on the container. 11:25.760 --> 11:34.000 And then the same thing for B2. And then this is the cool part. This is where we're defining 11:34.000 --> 11:39.840 our build. So, before we have base images, we're starting with Ubuntu and REL. And then we 11:39.840 --> 11:48.480 define our feature sets. So, here we have A1 and A2. And they're in the same level of the array, 11:48.480 --> 11:52.640 which means that those are conflicting features. We can also specify the versioning there 11:52.640 --> 11:59.520 if we needed to. And then we have B1 and B2 also as a set of features. And so, this is telling 11:59.520 --> 12:07.840 touch granular to build all possible combinations of either A1 or A2 with either B1 or B2 12:07.840 --> 12:13.280 on all versions of Ubuntu or REL. So, with just this small bit of configuration, we've defined 12:13.280 --> 12:20.320 a lot of images. So, here we'll show a brief example. Again, so here's our tree. We have A1 with 12:20.320 --> 12:26.400 the version files, A2 with that's version files. And then we have Wrangler.com, just going over 12:26.400 --> 12:31.120 the configuration we already showed in the other slide. I was just copying and pasted from this 12:31.120 --> 12:36.240 configuration file. Then we call touch granular update again. We fetch the version for Ubuntu 12:36.240 --> 12:43.120 Jami. Then we'll fetch it for REL. And then since we didn't have any other GitHub-based projects, 12:43.120 --> 12:48.320 we don't have to fetch the versions for those. So, once we fetch those two images, then we'll 12:48.400 --> 13:02.480 fetch the digest. And then we'll have a lock file. And so, after we have our lock file, we'll 13:02.800 --> 13:23.920 start. So, once we have our lock file, we can look at Wrangler.text again. And this lists out 13:23.920 --> 13:27.920 all of the images we're creating. So, from just those little bits of line of configuration, 13:27.920 --> 13:33.040 we've defined all of these images. Now, this is an extreme example. Hopefully, 13:33.040 --> 13:37.440 whatever team you're providing images, too, doesn't need this highly of a configuration. 13:37.440 --> 13:41.760 But, because of the way we design touch granular, this is possible. And not only possible, it's easy. 13:43.040 --> 13:48.880 Next, we're going to show that we're currently using Docker behind the scenes. We're 13:48.880 --> 13:53.600 looking at other options like Stacker, possibly something else. And this is just showing the 13:53.600 --> 13:58.480 Docker file that's generated from the lock file. So, for every different possible combination, 13:58.480 --> 14:05.120 we are defining the installation steps using our configuration file. 14:15.680 --> 14:20.880 And so, like we said last, we're creating a huge set of images with just a minimal amount of 14:20.880 --> 14:25.920 configuration. They're highly configured. Based on whatever the teams needs are, we can 14:25.920 --> 14:33.360 configure different runtime layers. We can configure the feature layers. We can use systems from 14:33.360 --> 14:40.560 with our local tree. And so, COVID is going to talk a little bit about our future direction, 14:40.560 --> 14:45.520 and then we'll open it up for some questions. And see, then. So, there's a reason. This demo 14:45.520 --> 14:50.800 was recorded. We are at the tail and self-opensourcing yet. We should have the project 14:50.800 --> 15:00.320 open source in the next two weeks or less. And the Apache 2.0. We do have, there's always 15:00.320 --> 15:05.760 work with CI integrations as we all know. So, there will be more CI CD works. It then brought up 15:05.760 --> 15:11.120 Sacker, the path partner and these are all things which we need to look into. Maybe integration 15:11.120 --> 15:18.800 to create.io. We want to build a community which has similar needs and work with us. So, get the 15:18.800 --> 15:24.800 dots out so that it is easier for foods to understand and consume. That's randler. And 15:24.800 --> 15:30.800 more other things which we would love to know and contributions through GitHub discussions 15:30.800 --> 15:35.840 once we have the project. And yes, testing, of course, we need to have more testing. 15:35.840 --> 15:38.080 That's us. Thank you. Questions? 15:48.960 --> 15:59.280 Hi. Thanks. Maybe I missed it. But one question is what language is Trucks Wrangler written in? 15:59.280 --> 16:05.280 Sure. And the second is, since you're building a bunch of images at lunch and you're saying 16:05.280 --> 16:09.440 you're doing some pipelines. What do you do? What do you do? Just one image fails but the 16:09.440 --> 16:15.200 rest succeed. How do you handle stuff like that? Awesome. Yeah. So, our Trucks Wrangler is built 16:15.200 --> 16:22.240 in Rust. We love Rust. It's got a great open source community and we thought it would be a great 16:22.240 --> 16:27.040 language to write this project in. And you had a great question about image building in consecutive 16:27.040 --> 16:33.120 or concurrent builds. We're currently looking at different ways to do concurrent builds within 16:33.120 --> 16:37.920 Tuxeringler. Currently, we are building out a Docker file and then building images from the 16:37.920 --> 16:43.040 Docker file. In the future, we imagine Tuxeringler will be responsible for building those images. 16:43.040 --> 16:48.000 And in that case, we'll have concurrent builds. And then the fail behavior will probably be 16:48.000 --> 16:52.480 configurable. 16:56.000 --> 17:01.440 So, if you have, if I understand correctly, you're creating a lot of Docker files right now. 17:01.440 --> 17:07.280 Currently, yes. Do you use buildings for this kind of stuff? 17:07.280 --> 17:12.000 Yes. So, currently, we build with BuildX. So, we can reuse our layers. We can specify the 17:12.080 --> 17:18.080 specific target layers that we need. This is our goal long-term. Long-term, we want to have 17:18.080 --> 17:23.360 everything being built inside Tuxeringler and abstract that from the user experience unless they 17:23.360 --> 17:30.320 want that. And because BuildX also provides you with the templates and something like that, 17:30.320 --> 17:36.080 I was wondering if you're trying to generate that for BuildX to be able to concurrently 17:36.560 --> 17:40.800 images and so on. Yeah. So, currently, we use a library called 17:40.800 --> 17:45.680 Ballard from Fuzzy Beaver that doesn't quite enable all of the features of BuildX 17:45.680 --> 17:50.880 we'd like. We're, while we're evaluating different building options like stacker, et cetera, 17:52.080 --> 17:56.720 build speeds is definitely a concern for us and as well as consistent builds. 17:58.320 --> 18:01.760 Were there any other questions? 18:02.640 --> 18:06.400 No. Okay. Seems like we're done for today. 18:06.400 --> 18:10.480 Oh, was there one up here? No. Okay. Yeah. If anyone has any questions, 18:11.200 --> 18:15.280 I have our emails I can go back to those on the first couple of slides. You can also 18:15.280 --> 18:23.200 feel free to reach out to either of us on LinkedIn. But thanks for everyone for coming. 18:23.200 --> 18:31.520 Yeah. And thanks for coming to the Container's Deaf Room and see you next year, hopefully.