WEBVTT 00:00.000 --> 00:09.000 So it's, oh, we have to wear this. 00:09.000 --> 00:14.000 It's great to be back. 00:14.000 --> 00:23.000 We see not standing front of the slide. 00:23.000 --> 00:29.840 So one of our marketing managers calls it the capital crime, not to introduce yourself. 00:29.840 --> 00:33.840 So that's me and one slide. 00:33.840 --> 00:39.840 I spent, I don't know how many years in product management must be 20 by now. 00:39.840 --> 00:45.840 And they're all in open source products, almost all. 00:45.840 --> 00:50.840 And before that, I was an engineer and a better engineer. 00:50.840 --> 00:57.840 And I was the maintainer of Manhattan, as you see at the bottom of the list of infamy. 00:57.840 --> 01:02.840 I wrote a book for a Riley, and that's why I'm surrounded by clouds in the picture. 01:02.840 --> 01:08.840 And that's kind of the short version. 01:08.840 --> 01:09.840 Hi, everybody. 01:09.840 --> 01:10.840 I'm Sage McTaggart. 01:10.840 --> 01:14.840 I use They Them pronouns, and I work on cybersecurity at IBM. 01:14.840 --> 01:18.840 I did my undergrad at UMass Amherst, and graduate school at UC Santa Cruz. 01:18.840 --> 01:23.840 I've done research in a lot of areas, ranging from programming languages, files, 01:23.840 --> 01:25.840 systems, tons of other stuff. 01:25.840 --> 01:29.840 And I've worked on security for Suffat Red Hat, and now IBM. 01:29.840 --> 01:35.840 I also work on incident response for Safanodef at IBM. 01:35.840 --> 01:44.840 And in my free time, I like to hike with my dog and garden and work towards a better world. 01:44.840 --> 01:52.840 So we're going to skip the interest lights because you know what Safin is. 01:52.840 --> 01:58.840 It's great, believe us, and Rook is even better. 01:58.840 --> 02:03.840 That's just go to the security points. 02:03.840 --> 02:08.840 So they raise expensive in so many ways when things go wrong. 02:08.840 --> 02:14.840 It's costly to collect, curate, storing it, ensuring regulatory compliance. 02:14.840 --> 02:16.840 And success is not optional. 02:16.840 --> 02:21.840 Failure to meet these requirements results in big fines at the least. 02:21.840 --> 02:28.840 Out of the spectrum, and it's loss of customers, loss of confidence. 02:28.840 --> 02:33.840 And services going down at the worst end of the spectrum. 02:33.840 --> 02:37.840 Actually going out of business would be the worst end of the spectrum, 02:37.840 --> 02:40.840 but let's not be too melodramatic. 02:40.840 --> 02:49.840 I like this interesting point at the end of the slide, which is, you want your systems, 02:49.840 --> 02:54.840 so that you are not the easiest target of those available. 02:54.840 --> 02:59.840 That goes for almost every product, and surprisingly, especially for an open source vendor. 02:59.840 --> 03:06.840 So many of my customers pay for upgrades, and then they don't install them. 03:06.840 --> 03:14.840 But anyway, so we're going to go over what are the best practices to secure Saf. 03:14.840 --> 03:21.840 And then also look at what's the newest, what's changed recently, and what we have improved. 03:21.840 --> 03:27.840 And here is a little bit of an agenda of what we're going to cover. 03:27.840 --> 03:31.840 So, track models. 03:31.840 --> 03:37.840 Security practices harden a specific point of the infrastructure. 03:37.840 --> 03:40.840 We have a security guide that's linked at the end, 03:40.840 --> 03:45.840 but this is a bigger point about how you use such a security guide. 03:45.840 --> 03:49.840 Cherry picking security practices without the model of the threat, 03:49.840 --> 03:53.840 and of the attacker, is not a viable strategy. 03:53.840 --> 03:59.840 The usual joke goes that if you want to protect from all possible threats, 03:59.840 --> 04:06.840 you need to turn the computer off, bury it in concrete, and then drop it at the bottom of the mariana trench. 04:06.840 --> 04:10.840 In other words, absolute security is not very usable. 04:10.840 --> 04:13.840 It needs to be defined in the context of the threat model. 04:13.840 --> 04:18.840 Are you facing script-key these, or the GRU, 04:18.840 --> 04:24.840 or the dry that privileged insider, is your share point administrator, 04:24.840 --> 04:27.840 siphoning off all of your files. 04:27.840 --> 04:31.840 These are very different threats. 04:32.840 --> 04:34.840 Some of these want to steal your data. 04:34.840 --> 04:39.840 Others want to crypto-lock you and hold you for ransom. 04:39.840 --> 04:43.840 Others may be just satisfied creating disruption. 04:43.840 --> 04:47.840 You need to define what threats you're protecting against, 04:47.840 --> 04:49.840 and pick your battles. 04:49.840 --> 04:56.840 If everything is a priority, which is something that some managers like to throw around, 04:56.840 --> 04:58.840 then nothing is. 04:58.840 --> 05:06.840 So, focus in the resources you have, where they can make the most impact. 05:06.840 --> 05:11.840 With that we've been moving beyond a simple inventory of regards points and hardening those, 05:11.840 --> 05:14.840 thinking of future threats to our software. 05:14.840 --> 05:18.840 This underlines our principles with security design. 05:18.840 --> 05:21.840 We're also working on general improvements, 05:21.840 --> 05:25.840 a key part of this is ensuring that we do not deviate from standard protocols. 05:26.840 --> 05:30.840 We do this because we trust the community, such as cryptographers, 05:30.840 --> 05:36.840 to have the best practices, or at least practices that are better than our ideas. 05:36.840 --> 05:38.840 Don't write your own crypto. 05:38.840 --> 05:41.840 Think you probably heard that one before. 05:41.840 --> 05:49.840 Let's now talk about one way we build security into our design, 05:49.840 --> 05:54.840 and recent improvements we made here. 05:54.840 --> 06:01.840 Here is the network layout, so how the different security sections of the network look like. 06:01.840 --> 06:05.840 The public security zone is an entirely untrusted area of the cloud. 06:05.840 --> 06:09.840 Could be the internet as a whole, just networks external to your cluster, 06:09.840 --> 06:11.840 that you have no authority over. 06:11.840 --> 06:15.840 Data transmissions crossing this zone should make use of encryption. 06:15.840 --> 06:18.840 Note that the public zone, as I just defined it, 06:18.840 --> 06:22.840 does not include the storage cluster front end, 06:22.840 --> 06:27.840 which in SEF is confusingly called the public underscore network, 06:27.840 --> 06:34.840 which defines the storage front end and properly belongs in the storage access zone. 06:34.840 --> 06:40.840 The SEF client zone refers to network networks accessing SEF clients, 06:40.840 --> 06:45.840 like the object gateway, the SEF file system, or block storage. 06:45.840 --> 06:50.840 SEF clients are not always excluded from the public security zone. 06:50.840 --> 06:55.840 For instance, it is common to expose the object gateway's front end to the liver, 06:55.840 --> 06:58.840 the SEF API in the public security zone. 06:58.840 --> 07:03.840 Next, the storage access zone is instead an internal network, 07:03.840 --> 07:09.840 providing SEF clients with access to the storage cluster itself. 07:09.840 --> 07:13.840 Finally, the cluster zone refers to the most internal network, 07:13.840 --> 07:17.840 providing storage nodes with connectivity for replication, 07:17.840 --> 07:21.840 heartbeat, backfill, and recovery tasks. 07:21.840 --> 07:28.840 This zone includes the SEF cluster back end network called the cluster network in SEF. 07:28.840 --> 07:32.840 Operators offer unclear text traffic in the cluster zone, 07:32.840 --> 07:38.840 relying on the physical separation or villan separation of the network from all other traffic. 07:38.840 --> 07:42.840 This would not be a valid choice going back to our threat model discussion. 07:42.840 --> 07:48.840 If your threat model includes adversarial privileged insiders. 07:48.840 --> 07:54.840 So again, what is the threat that you want to encrypt your internal communication 07:54.840 --> 08:01.840 because somebody can go and tap your rack that should be part of your model. 08:01.840 --> 08:10.840 These four zones are separately mapped or combined depending on the use case and the design of your specific cluster. 08:11.840 --> 08:19.840 Components spanning the boundary of two security zones with different trust or authentication requirements must be carefully configured. 08:19.840 --> 08:27.840 These are natural weak points in network architecture and should always be configured to meet requirements of the higher trust level of the zones connected. 08:27.840 --> 08:39.840 Many cases the security controls should be a primary concern due to the likelihood of somebody probing to escalate from a lower privileged zone to a higher privileged zone. 08:41.840 --> 08:54.840 Operators in general should consider exceeding zone requirements at integration points, which for a storage product is often easier to accomplish than for let's say compute platform. 08:54.840 --> 09:03.840 For example, the cluster security zone can be isolated from other zones easily because there is no reason for it to connect to other zones. 09:03.840 --> 09:19.840 Conversely, an object gateway in the client security zone will need to access the cluster security zones monitors OSDs all of them and will likely expose its S3 API to the public security zone on ports 80 and 443. 09:19.840 --> 09:27.840 So this one is talking to three different zones at least. 09:27.840 --> 09:36.840 Some engineers on the team have worked to improve the dashboard security bringing it all within a single security zone. 09:36.840 --> 09:56.840 So the change here is that previously only, only authentication to the dashboard was secured and in theory you could have gone and tapped the information, the monitoring information of the cluster from previous or graphana. 09:56.840 --> 10:10.840 Now everything all of the web sockets are secured uniformly and there is a reverse proxy in front which has other engineering benefits that are not security related. 10:10.840 --> 10:25.840 And that is the biggest change in this kind of in the network part of our security practices. 10:25.840 --> 10:34.840 Yeah, that about sums that one up. It's been a fantastic improvement to the dashboard security as of squid. 10:34.840 --> 10:36.840 Going on to our next slide. 10:36.840 --> 10:42.840 We recently moved well three years ago almost from Red Hat to IBM. 10:42.840 --> 10:50.840 And you know the security of a product really it depends a lot on who's maintaining it. 10:50.840 --> 10:55.840 Some open source products are not maintained by anybody or there's one person. 10:55.840 --> 10:58.840 They maintain it in their basement in their free time. 10:58.840 --> 11:06.840 That's fantastic, but that isn't necessarily practical for an industry or any sort of case where we're very worried about security. 11:06.840 --> 11:15.840 So I'm going to discuss some aspects of IBM product security and how that's been going, especially with the implications for open source and upstream staff. 11:15.840 --> 11:21.840 And also a little bit of compliance as open source since a lot of compliance models are not designed for open source. 11:21.840 --> 11:28.840 I'll discuss some of our accomplishments and what we're planning on doing going forward in the next year. 11:28.840 --> 11:33.840 So product security at IBM, we follow a secure development lifecycle. 11:33.840 --> 11:40.840 This is the, this is done with the goal of reducing risk and improving security for both South and and Rook. 11:40.840 --> 11:49.840 We suggest improvements, we pen test, we manifest all of our dependencies, we review vulnerabilities and track weaknesses and exploits. 11:49.840 --> 11:56.840 We check every new release for these things and we just generally are trying to iterate and do better. 11:56.840 --> 12:05.840 We don't release with known criticals unless we get an exemption, which is pretty rare, that reduces the exposure to different CVEs. 12:05.840 --> 12:15.840 And as a result, we're improving upstream staff because all of these get ported back to the non IBM staff. 12:15.840 --> 12:25.840 We still work on secure design there and we've worked also with IBM to implement open source principles by improving things like their call home feature. 12:26.840 --> 12:35.840 And we're working with IBM research on how to implement TCMs and where open source quantum resistant algorithms might go. 12:35.840 --> 12:42.840 So one thing that we do a lot of is we manifest and document dependencies within South. 12:42.840 --> 12:45.840 We automate security scans of our code and containers. 12:45.840 --> 12:54.840 We work heavily with upstream communities and this has sort of been a change since moving to IBM to bump dependencies and ensure that fixes are made for all affected projects when possible. 12:54.840 --> 13:05.840 The scanning benefits the entire open source community because we'll say, hey, Kephanna, how's your go-line version and they update it and they're pretty good on their CVEs. 13:05.840 --> 13:12.840 So we work together with different open source projects to make sure that they're getting updated whenever we notice something. 13:12.840 --> 13:17.840 This enables us to be better contributors to the open source community. 13:18.840 --> 13:33.840 And these dependencies are need for upgrades they cover the vast majority of our CVEs which is a little bit of a pain but updating to the latest version and automating that as much as possible and making as easy as possible has really helped us. 13:33.840 --> 13:38.840 So it's really hard to automatically update your dependencies you're going to run into build issues. 13:38.840 --> 13:42.840 Yeah, sure, depend upon on GitHub says update your dependency. 13:42.840 --> 13:47.840 You do that, it might start breaking things, you might have to refactor your code, et cetera, et cetera. 13:47.840 --> 13:52.840 Especially if it's been in our case, here's since we've upgraded some of the more obscure dependencies. 13:52.840 --> 14:00.840 And I'm sure all of us are in that boat where that's happened where we notice we have to update something and if you have thousands of dependencies, that's what happens. 14:01.840 --> 14:09.840 So what we do there is we're trying to reduce our attack surface in the components that have known CVEs with redesigns like we saw with the dashboard. 14:09.840 --> 14:22.840 And we're also applying what I'm going to call a humination approach, human automation, where we create a very clear, simple, almost automatic process at every release saying this dependency. 14:22.840 --> 14:27.840 And I'm going to blame Grafana again, even though they're a lovely project, they're wonderful people. 14:27.840 --> 14:33.840 We're going to say, hey, Grafana has a lot of our CVEs that get reported to us. 14:33.840 --> 14:35.840 Let's update Grafana every release. 14:35.840 --> 14:41.840 That way we're only updating between a few releases at each time, we're following a clear process, we know when we're doing that. 14:41.840 --> 14:51.840 And we start with our worst offenders, and then we continue updating all of them until they're all relatively recently updated and we have clear processes. 14:51.840 --> 14:54.840 And that's been a major improvement that we've done. 14:55.840 --> 14:56.840 Whoops, sorry. 14:56.840 --> 15:02.840 So now briefly going into executive order, one for zero to eight. 15:02.840 --> 15:06.840 This was done by President Biden back in 2021. 15:06.840 --> 15:10.840 Scanners sometimes, sorry. 15:10.840 --> 15:13.840 Sorry, let me get on the right side of my apologies. 15:13.840 --> 15:17.840 Scanners oftentimes only detect CVEs from dependencies. 15:17.840 --> 15:23.840 This was an executive order that basically required us to start manifesting and documenting our CVEs. 15:23.840 --> 15:26.840 And also all of our dependencies. 15:26.840 --> 15:35.840 So we can debate what is actually required, what isn't, but for our purposes, we need to know all of our dependencies and all known vulnerabilities and update them. 15:35.840 --> 15:40.840 So we use scanners and they detect a lot of CVEs. 15:40.840 --> 15:44.840 Open source has a lot of CVEs in a lot of cases, and that's a good thing. 15:44.840 --> 15:46.840 We know our CVEs. 15:46.840 --> 15:49.840 We ran these scanners on closed source code. 15:49.840 --> 15:51.840 You would find even more CVEs. 15:51.840 --> 16:01.840 But you tell somebody, hey, you can't have CVEs or you have to have a plan to fix all CVEs or you have to evaluate it all CVEs in all of your dependencies. 16:01.840 --> 16:04.840 You run these scanners on them. 16:04.840 --> 16:06.840 They're going to give nasty results. 16:06.840 --> 16:09.840 And you're going to discourage people from using open source. 16:09.840 --> 16:15.840 So how do we resolve that with this emphasis on scanning that gets all of these? 16:15.840 --> 16:18.840 Incident responses usually focused on are you vulnerable? 16:18.840 --> 16:19.840 Do you have a proof of concept? 16:19.840 --> 16:21.840 Is there an exploit? 16:21.840 --> 16:28.840 Well, we really need to work on redesigning things to reduce our attack surface, so we have time to fix these. 16:28.840 --> 16:30.840 We have to look at things like the vex scoring. 16:30.840 --> 16:34.840 And we have to update dependencies to reduce the burden later. 16:34.840 --> 16:44.840 So as we know all of our vulnerabilities from these scanners, as opposed to just what we're guessing our vulnerabilities, we can say, let's redesign and fix this. 16:44.840 --> 16:55.840 For example, at IBM, they have a feature called call home that basically helps them provide IT support almost and provide telemetry and other data metrics. 16:55.840 --> 16:57.840 So it was not open source. 16:57.840 --> 16:59.840 It was not designed in an open source way. 16:59.840 --> 17:03.840 The security model originally required closed source code. 17:03.840 --> 17:09.840 We were able to use our incident response skills to create a more secure one advocating for open source in the project. 17:09.840 --> 17:13.840 Those both redesigned to be less vulnerable have a smaller attack surface. 17:13.840 --> 17:24.840 And we were able to do that because we were aware of our potential vulnerabilities from all of our incident response work with open source. 17:24.840 --> 17:35.840 So you really want to use code that's up to date, but all this scanning has really helped us and brought issues to light and given us ways to fix it. 17:35.840 --> 17:42.840 So again, as part of this, we eventually want to start automatically updating dependencies. 17:42.840 --> 17:52.840 But as I've touched on before, that can present challenges creating communication is definitely a very viable way to go and what we're doing. 17:52.840 --> 18:04.840 And we've done things also including redesigning our upstream CV e-reporting to account for different processes to just try to keep things flowing in a very clear systemic and easy way. 18:04.840 --> 18:07.840 That reduces the burden on the people behind the project. 18:07.840 --> 18:13.840 And what does this matter? Well, we might have government customers that need this. 18:13.840 --> 18:17.840 We might have corporate environments that say we don't want to use poorly maintained code. 18:17.840 --> 18:25.840 And also, to really want to use code where your dependencies are years out of date, you probably don't. 18:25.840 --> 18:32.840 So again, this automation process has really helped us a lot. 18:32.840 --> 18:39.840 We do run into issues with things like transitive dependencies. Let's say we're using a dependency that uses going. 18:39.840 --> 18:46.840 And I'm just going to use going as an example, because whenever they do a release, they're fixing CV e's and almost all cases. 18:46.840 --> 18:51.840 Well, a strict reading would say we have to treat that the same and handle the automation accordingly. 18:51.840 --> 18:58.840 So that's where we're trying to make sure that we're a good partner with the broader open source community when our scanners find this. 18:58.840 --> 19:03.840 And isn't just hidden in corporate IBM land, but instead we're saying, hey, this found this. 19:03.840 --> 19:06.840 Are we updating this? How can we work on this process? 19:06.840 --> 19:14.840 And this has really helped us a lot, especially for a very small security and build team and made it a lot less manual. 19:14.840 --> 19:21.840 And I think we're a lot of open source communities we're relying on volunteer time. This helps really reduce our burden here. 19:21.840 --> 19:25.840 And just the processes make it just so much easier on all of us. 19:25.840 --> 19:29.840 So now let's briefly talk about how Seth does encryption. 19:29.840 --> 19:35.840 Server side operators overwhelmingly choose to encrypt data at risk using the Linux simplified key mechanism. 19:35.840 --> 19:43.840 Lux all data and metadata on a storage cluster can be secured using a variety of DN crypt configurations. 19:43.840 --> 19:46.840 And almost all of our customers choose to do this. 19:46.840 --> 19:50.840 It's possible that somebody doesn't, but we recommend it. It's standard. 19:50.840 --> 19:56.840 We also enable a security best practice by locating our mons on separate hosts from the OSBs. 19:56.840 --> 20:00.840 The centers to anti affinity of the keys and the data they encrypt. 20:00.840 --> 20:04.840 So if you see a one drive, you're not getting all the information and the key. 20:04.840 --> 20:12.840 The objects for gateway has additional capabilities, including encryption at ingestion time, the use of per user keys as opposed to per. 20:12.840 --> 20:17.840 Drive keys, key rotation with tools like vault, support for AWS. 20:17.840 --> 20:26.840 SSC KMS and more. We also use Department of Defense 140 ciphers when supplied by rel appropriately. 20:26.840 --> 20:30.840 And we also encrypt our data and transit. 20:30.840 --> 20:37.840 We do this by turning on the Seth protocol communication for messenger version 2.1 with not allus. 20:37.840 --> 20:43.840 We, we note that a lot of people might want to use clear text as a compatibility reasons. 20:43.840 --> 20:48.840 There really isn't much overhead, so that's not necessarily the reason, but if you have a legacy. 20:48.840 --> 20:52.840 If you have a need for the legacy clear text protocol, some people do. 20:52.840 --> 20:59.840 We allow that. We also obviously encrypt all data at rest if you choose to. 20:59.840 --> 21:08.840 And we're working on redesigning our data in transit model for quantum resistance as algorithms have been now published that are recommended by nist. 21:08.840 --> 21:11.840 We'll be seeing what we can do there. 21:12.840 --> 21:19.840 Again, we have the client and public security zones. We have TLS security for the object gateway to us three clients. 21:19.840 --> 21:24.840 The TLS termination at HA Proxies a special case. 21:24.840 --> 21:36.840 And that was recently done with squid to ensure that the link is in the appropriate security zone. 21:36.840 --> 21:43.840 And then, of course, with the network standard practices like firewalling individual nodes, 21:43.840 --> 21:46.840 to expose only clear list of ports supplies. 21:46.840 --> 21:56.840 So now going into rook specifics, rook can use CRD's custom resource definitions along with several other options to encode many of these features, 21:56.840 --> 22:02.840 such as configuring the trust certificates for the rate of skateway web server or GW. 22:03.840 --> 22:11.840 Rook supports at rest data encryption as discussed earlier with the in-flight stuff protocol encryption as an option in 1.9. 22:11.840 --> 22:15.840 Kubernetes permission system applies to our persistent volume. 22:15.840 --> 22:18.840 So permissions, quotas, all that comes from Kubernetes. 22:18.840 --> 22:20.840 Nothing rook needs to do here. 22:20.840 --> 22:23.840 Rook supports, of course, key management systems. 22:23.840 --> 22:26.840 And this allows individual volumes being encrypted with their own key. 22:26.840 --> 22:32.840 All of this basically results in making it really easy for us to follow best practices such as key rotation, 22:32.840 --> 22:35.840 replication, and limiting scope for each key. 22:35.840 --> 22:40.840 This limits the scope of unencrypted traffic, and this limits the scope of each key. 22:40.840 --> 22:47.840 So going briefly into the control plane, as popularized by Ansible, SSH is used by Suff admin, 22:47.840 --> 22:50.840 Suff Ansible, and other deployment and day one tools. 22:50.840 --> 22:55.840 To provide a secure command line path for install and upgrade installation as part of host management, 22:55.840 --> 23:00.840 we do this using the dashboard so that the dashboard is not usually exposed to the world, 23:00.840 --> 23:05.840 and yet it can still be accessed by the operator's workstation to be abuse. 23:05.840 --> 23:10.840 We recently, again, did the redesign that puts it all within one security zone for the dashboard, 23:10.840 --> 23:16.840 and the manager also then supports the whole infrastructure and needs to be reachable on the storage access zone. 23:16.840 --> 23:19.840 Hence, why we're using SSH. 23:19.840 --> 23:23.840 So we're always trying to improve our cryptography. 23:23.840 --> 23:25.840 We are not cryptographers. 23:25.840 --> 23:31.840 We can read stuff that cryptographers are doing and think that it's fantastic though. 23:31.840 --> 23:37.840 So some things that we've been doing is we've been talking with IBM research about open source quantum computing and 23:37.840 --> 23:38.840 confidential computing. 23:38.840 --> 23:43.840 We're currently determining the viability where we plug things in, et cetera. 23:43.840 --> 23:49.840 We would use open source code that from the open source quantum project, 23:49.840 --> 23:53.840 and we're documenting all the places where we use cryptography. 23:53.840 --> 23:58.840 Seeing if it's quantum safer, we've done the initial assessment that basically determined, 23:58.840 --> 24:03.840 hey, we're using AES, that's safer, but it isn't what is recommended. 24:03.840 --> 24:08.840 So now we're going forward with that and also seeing where we could plug in things like TCMs. 24:08.840 --> 24:12.840 So our priority here have it all be open source, use open source libraries. 24:12.840 --> 24:15.840 Don't have it become proprietary to be quantum safe. 24:15.840 --> 24:19.840 We're working with upstream to discuss the correct use of cryptographic modules, 24:19.840 --> 24:21.840 such as AES. 24:21.840 --> 24:24.840 Of course, we use AES, but how are we implementing it? 24:24.840 --> 24:25.840 What type of model are we using? 24:25.840 --> 24:28.840 Should we use GCM, different models, et cetera. 24:28.840 --> 24:33.840 So we're working with our upstream community to ensure that we're implementing, 24:33.840 --> 24:38.840 according to the Kerbros model, not just rolling our own ideas of where we should plug in AES. 24:38.840 --> 24:41.840 We're tying that in with the need to upgrade dependencies automatically, 24:41.840 --> 24:44.840 because you need to upgrade your cryptographic libraries too. 24:44.840 --> 24:47.840 They can have vulnerabilities. They are an infallible. 24:47.840 --> 24:50.840 Anybody can put a bug in code anywhere. 24:50.840 --> 24:54.840 So talking a little bit about identity and access, 24:54.840 --> 24:58.840 Seth's use of shares keys prevents mint from clusters from having 24:58.840 --> 25:00.840 man in the middle attacks by default. 25:00.840 --> 25:04.840 You still want to have good practices, such as key ring, read and write. 25:04.840 --> 25:09.840 You want to only have those for your current user and route. 25:09.840 --> 25:14.840 And your client admin user should be really route only. 25:14.840 --> 25:17.840 You don't really want all users to be route. 25:17.840 --> 25:19.840 This prevents that, which is the best practice. 25:19.840 --> 25:21.840 Talking briefly about RGW. 25:21.840 --> 25:24.840 It supports the key and secret model of AWS S3, 25:24.840 --> 25:27.840 and the equivalent model for OpenStack Swift. 25:27.840 --> 25:31.840 Again, we want to treat that administrator key with appropriate respect. 25:31.840 --> 25:35.840 Use administrator user sparingly to reduce your risk profile. 25:35.840 --> 25:38.840 Your RGW user data is stored in Seth pools, 25:38.840 --> 25:42.840 which should be secured again as we discussed previously regarding users at risk. 25:42.840 --> 25:46.840 We can couple with OIDC providers such as key cloak, 25:46.840 --> 25:51.840 which you can back with your, you basically can just use SSO here. 25:51.840 --> 25:57.840 With your organizational IDP, or again, just use SSO. 25:57.840 --> 26:01.840 And we support LDAP for identity and access. 26:01.840 --> 26:07.840 We recommend security LDAP along with options to use SSO as needed. 26:07.840 --> 26:11.840 And this has sort of been a recent improvement in Seth. 26:11.840 --> 26:13.840 So we're using SSO more. 26:13.840 --> 26:17.840 Just to standardize it and keep it basically per user. 26:17.840 --> 26:22.840 And one state has deleted from a Seth cluster though. 26:22.840 --> 26:25.840 We do have some practices around data retention. 26:25.840 --> 26:27.840 One state has deleted from a Seth cluster. 26:27.840 --> 26:29.840 It cannot be recovered for practical use. 26:29.840 --> 26:31.840 There's some exceptions. 26:31.840 --> 26:35.840 RBD has a new facility called Trashbin. 26:35.840 --> 26:39.840 And again, in RGW, there's some specific cases where 26:39.840 --> 26:43.840 the versioning of store buckets may result in deleted objects being preserved 26:43.840 --> 26:45.840 until their purge per policy. 26:45.840 --> 26:49.840 So again, configure your data storage pools accordingly. 26:49.840 --> 26:53.840 So again, there's a lot of things that you can do there to just keep it easier. 26:53.840 --> 26:58.840 And to sanitize your retired medium, you want to encrypt your OSD contents at 26:58.840 --> 27:00.840 rest and then destroy the encryption key. 27:00.840 --> 27:02.840 That's probably your quickest way to do it. 27:02.840 --> 27:06.840 You can definitely handle it all their ways, but this is the quickest way. 27:06.840 --> 27:08.840 And it works. 27:08.840 --> 27:14.840 So going into infrastructure hardening, the hardening options are very 27:14.840 --> 27:16.840 vendor-dependent. 27:16.840 --> 27:21.840 We have some choices here with the red hat modes here. 27:21.840 --> 27:24.840 We have SC Linux, of course. 27:24.840 --> 27:28.840 We have PIPs 140 slash 2 support. 27:28.840 --> 27:31.840 We definitely have our certified cryptographic options 27:31.840 --> 27:34.840 to import this in FIPs mode setup. 27:34.840 --> 27:37.840 And, you know, relate.force most recent certified version. 27:37.840 --> 27:39.840 There's sometimes a delay. 27:39.840 --> 27:42.840 So, you know, we use that version when needed. 27:42.840 --> 27:44.840 We have some hard and binaries. 27:44.840 --> 27:49.840 We have additional kernel or OS supplied hardening as well. 27:49.840 --> 28:03.840 We have a few links for you about things that we have referenced or 28:03.840 --> 28:06.840 indirectly mentioned. 28:06.840 --> 28:10.840 You can get from the slides when we distribute that. 28:11.840 --> 28:13.840 Yep. 28:17.840 --> 28:19.840 I'm sorry you had a question? 28:19.840 --> 28:20.840 Yes. 28:20.840 --> 28:22.840 You did not talk about the set of S. 28:22.840 --> 28:24.840 What's the ACL option? 28:24.840 --> 28:26.840 What's the set of S that you had? 28:26.840 --> 28:28.840 Excess control this set of S. 28:28.840 --> 28:30.840 That's the security topic for me. 28:30.840 --> 28:35.840 Yeah, we didn't go there because they generally are our standard 28:35.840 --> 28:36.840 posics. 28:36.840 --> 28:38.840 We don't really diverge much there. 28:38.840 --> 28:39.840 I don't think. 28:39.840 --> 28:42.840 So, the question was, what are the access control lists in 28:42.840 --> 28:43.840 SAPFS? 28:43.840 --> 28:44.840 Yeah, yes. 28:44.840 --> 28:45.840 Yeah. 28:45.840 --> 28:47.840 It's just standard. 28:47.840 --> 28:49.840 All these permissions or public's ACLs, 28:49.840 --> 28:52.840 or the best before ACLs is PIPs. 28:52.840 --> 28:54.840 For example, that was at ZFS. 28:54.840 --> 28:55.840 Posics. 28:55.840 --> 28:56.840 Okay. 28:57.840 --> 28:58.840 Posics. 29:07.840 --> 29:09.840 There's nothing. 29:09.840 --> 29:10.840 Go ahead. 29:10.840 --> 29:12.840 Who checks the posics? 29:12.840 --> 29:13.840 Actors? 29:13.840 --> 29:16.840 Is it the safe client or is it the safe server? 29:16.840 --> 29:19.840 I believe it's the client. 29:19.840 --> 29:23.840 And sorry, the question was who checks the posics ACLs? 29:23.840 --> 29:24.840 Okay. 29:24.840 --> 29:27.840 We need one of the developers, and I don't see him here. 29:27.840 --> 29:29.840 But, oh, there we go. 29:29.840 --> 29:32.840 Patrick, go ahead. 29:32.840 --> 29:34.840 It's the client. 29:34.840 --> 29:35.840 All right. 29:46.840 --> 29:48.840 Outside. 29:48.840 --> 29:50.840 Thank you. 29:54.840 --> 29:55.840 Thank you.