WEBVTT 00:00.000 --> 00:11.840 All right, welcome and one, and I hope you still have some energy left because I have about 00:11.840 --> 00:21.400 60 slides for 30 minutes, and you're done, okay I'm done. 00:21.400 --> 00:26.560 As I love the energy, thank you so much for still being here at the end of Phasm, I was told 00:26.560 --> 00:28.200 to stand over here. 00:28.200 --> 00:35.360 We just heard that sometimes we need to make the decision to fork, fix or forgo, and how 00:35.360 --> 00:39.560 do we do this at scale, and I want to talk with you about that. 00:39.560 --> 00:45.600 One of the thesis that I have, so this is the punchline, is that tracking open source 00:45.600 --> 00:49.200 project health is proactive risk management. 00:49.200 --> 00:54.200 When I look at the health of open source projects at the potential to continue developing 00:54.200 --> 01:01.240 quality software and risk management to look for reasons why that might not be the case. 01:01.240 --> 01:09.000 And I think project health is a place we need to look because what is happening inside 01:09.000 --> 01:17.320 of an open source community today will determine what the software looks like down the road, 01:17.320 --> 01:19.920 whether it's still maintained or not. 01:19.920 --> 01:26.640 And so I'm gear link, I'm in this space, I found that the chaos project I've been 01:26.640 --> 01:34.840 around for some time, and really want to help make open source across the healthy and sustainable. 01:34.840 --> 01:40.920 Here is what I have in store for you across the next 60 slides, being perplexed by open 01:40.920 --> 01:45.240 source in the software supply chain as well as we'll start. 01:45.240 --> 01:52.400 When we look at open source across the board, we know that about 10% of everything that 01:52.400 --> 01:58.400 gets published gets used, only about 10% of everything that that's published is actually 01:58.400 --> 01:59.400 maintained. 01:59.400 --> 02:05.520 So there's a lot of noise in open source, and there are bad people who have found this 02:05.520 --> 02:13.000 to be very attractive attack vector with more and more malicious packages being published 02:13.000 --> 02:14.960 and discovered. 02:15.000 --> 02:20.400 This is costing the industry 60 billion as of today, approximately. 02:20.400 --> 02:27.840 And any company that thinks, it's not me, about 45% of companies I expect to have been 02:27.840 --> 02:32.240 hit by a software supply chain attack, according to Godner. 02:32.240 --> 02:35.280 Well, what's the impact of that? 02:35.280 --> 02:42.760 From the Sullivan example, we know it was about 11% of annual revenue to mitigate this software 02:42.760 --> 02:44.240 supply chain attack. 02:44.280 --> 02:47.600 So this is something that is bugging us. 02:47.600 --> 02:50.000 So now we're in the Espanthew Room. 02:50.000 --> 02:57.320 We can start to articulate what it is that we have to worry about. 02:57.320 --> 03:03.920 And I want to use a metaphor of software, a just more like milk than wine. 03:03.920 --> 03:04.920 Why is this relevant? 03:04.920 --> 03:11.480 When we look at milk products or dairy products, we know exactly where that is being 03:11.480 --> 03:12.480 used. 03:12.480 --> 03:17.280 The industry has already established that we need to track the source. 03:17.280 --> 03:25.480 And now with the CRA as the latest example of why we need to do this, we need to do our 03:25.480 --> 03:30.640 due diligence and actually check what it is that we have in our dependencies. 03:30.640 --> 03:37.840 So as an industry, there has been a move towards uncovering what are we actually using and 03:37.840 --> 03:46.120 documenting this and with 70% to 95% of software containing open source, we really need 03:46.120 --> 03:48.480 to focus on the open source part. 03:48.480 --> 03:54.760 And we also need to look at the transitive dependencies because if we don't, then we have 03:54.760 --> 04:01.040 things in our software that we don't know about and so we have unknown risk there. 04:01.040 --> 04:04.520 So this is where the software bill of materials come in. 04:04.520 --> 04:13.400 Because our inventory of open source software that we are relying on and other components. 04:13.400 --> 04:18.880 There was a really brief overview of why we are here talking about S-bombs and why I want 04:18.880 --> 04:22.400 to talk with you about project health. 04:22.400 --> 04:28.280 Now that we have S-bombs across the industry and I know from talking to many companies 04:28.280 --> 04:31.400 that there is a lot of work in this space. 04:31.400 --> 04:38.240 We can do new things, we can do new kinds of analyses and give you another metaphor. 04:38.240 --> 04:45.080 Think of driving a car, driving down the road and the fuel gauge comes on. 04:45.080 --> 04:49.280 You know what to do, you pull over to the gas station, you pull up, it's good to go. 04:49.280 --> 04:51.000 The engine light comes on. 04:51.000 --> 04:54.920 You go to a mechanic, you get it repaired, no problem. 04:54.920 --> 05:01.000 So long as you are using a modern car, once there are 20 years old or so you start going 05:01.000 --> 05:05.440 into the junkyard for replacement parts and you have to look for a specialist. 05:05.440 --> 05:12.320 And when we transfer this to software, we have a similar situation where we can look at what 05:12.320 --> 05:20.040 we have today, scan for licenses, scan for vulnerable security, and we can fix that. 05:20.040 --> 05:26.920 But when we don't pay attention to whether there is a community around, maintaining the software, 05:26.920 --> 05:34.120 then we might run into that problem of having un-maintained unreliable software. 05:34.120 --> 05:42.920 And so I am here to propose that we have these three pillars that we need to be able 05:42.920 --> 05:46.520 to trust our software dependencies. 05:46.520 --> 05:53.840 And with licenses, we have license scanners with security, we have software composition analysis, 05:53.840 --> 06:02.120 we have vulnerability databases, for project health, we have the community, chaos community 06:02.120 --> 06:05.360 to talk about how we should look at project health. 06:05.360 --> 06:10.160 But I don't really see a lot of uptake on that yet across the industry. 06:10.160 --> 06:16.800 I think that third pillar of project health is still overlooked in the open source strategy 06:16.800 --> 06:18.920 by many. 06:18.920 --> 06:26.200 And so this is where I want to think about how can we anticipate the risk by looking at project 06:26.200 --> 06:28.400 health and how do we do this? 06:28.400 --> 06:35.680 Remember the beginning when I said I had a thesis that looking at project risk, a project 06:35.680 --> 06:40.440 health is a way to proactively manage the risk. 06:40.440 --> 06:42.760 Well, now we are back to this. 06:42.760 --> 06:43.760 How do we do this? 06:43.760 --> 06:46.120 How do we operationalize this? 06:46.200 --> 06:52.000 I want to propose that we look at whether a project is maintained with some metrics that 06:52.000 --> 06:55.320 we can look at at scale. 06:55.320 --> 07:01.360 Look at the community handle the demand or the work they might have work that they have. 07:01.360 --> 07:09.040 Do they have the capacity to quickly respond to issues and pull requests and resolve 07:09.040 --> 07:10.040 them? 07:10.040 --> 07:15.440 And do they have the talent, the people in the community, whether it is through new people 07:15.440 --> 07:24.280 coming in, people staying long time or that we have at least more than one person responsible 07:24.280 --> 07:26.280 for maintaining it. 07:26.280 --> 07:30.760 So there are several things that we can automate from all the metrics that we could look 07:30.760 --> 07:32.720 at just as a starting point. 07:32.720 --> 07:36.960 I'm not saying this is the only set of metrics that matter, but as a starting point 07:36.960 --> 07:40.960 to start looking at project health. 07:40.960 --> 07:45.440 And then we can start scaling this. 07:45.440 --> 07:49.960 And this is where really S-bombs come in because when we look at project health, we've 07:49.960 --> 07:57.560 done this community by community so far, but now that we have inventories of tens of thousands 07:57.560 --> 08:02.600 of components that we care about, we want to do this at scale. 08:02.600 --> 08:12.000 And what if we take our list and calculate the health for each component? 08:12.000 --> 08:19.760 But if we assign each one a single score, take the seven metrics, normalize them, put them 08:19.760 --> 08:27.120 together with whatever algorithm we want to use, what else can we do with that? 08:27.520 --> 08:33.360 But once we have that, we can look across all of the components we use, we can filter 08:33.360 --> 08:39.680 by the different packages that we are building, the different solutions that we are building, 08:39.680 --> 08:46.040 and we can calculate the score, at least start to filter where do we need to pay attention. 08:46.040 --> 08:49.840 And so I want to show you a prototype that we've built. 08:49.840 --> 08:58.600 We've tried this out with the use case, and the data I'm going to show you is from 08:58.600 --> 08:59.600 Kubernetes. 08:59.600 --> 09:07.200 We've tried this also inside of a company already, but here's what it looks like. 09:07.200 --> 09:13.840 We built the dashboard, so this is for the use case of a controller or someone who wants 09:13.840 --> 09:16.120 to see, okay, I want to analyze. 09:16.200 --> 09:20.960 There's of course another use case of embedding it into the development workflow and 09:20.960 --> 09:28.080 CICD pipeline, but this is more of the background, I want to see the whole picture. 09:28.080 --> 09:35.680 So once I filter down to the project that I really care about, I can see how many of the 09:35.680 --> 09:42.680 dependencies fall in the buckets of low, medium, high and very high risk. 09:42.680 --> 09:49.480 I could color these in the future so that we have a bright, red category, and this helps 09:49.480 --> 09:56.520 us focus our attention on, okay, these are the dependencies that I want to spend some time 09:56.520 --> 09:58.640 investigating. 09:58.640 --> 10:05.920 And I still have the original data on what each of those metrics returned. 10:06.920 --> 10:12.560 We'll see this in a moment when we look at a particular package or component. 10:12.560 --> 10:23.080 When we drill down, we can see the overall score, but then we can also see why we got this 10:23.080 --> 10:24.080 score. 10:24.080 --> 10:31.160 And on the right, you see all the metrics, the raw data that we got from analyzing the 10:31.160 --> 10:37.680 open source community, and then our assessment on whether that is low medium high. 10:37.680 --> 10:45.160 So there are thresholds that we can parametrize, we can build our own thresholds here. 10:45.160 --> 10:51.160 And so for this one, we can say, okay, medium lead time to pull requests and issues 10:51.160 --> 10:57.360 is where the problem seems to be in this particular component. 10:57.360 --> 11:06.320 So now we have the data to make the decision, we started with, do we fix fork or fork? 11:06.320 --> 11:11.880 So if we want to help with the median lead time of pull requests and issues, well maybe 11:11.880 --> 11:17.600 we go and we help with buck triaging and make sure the right people get attention on the 11:17.600 --> 11:22.400 issues and pull requests and maybe that is the way we can help. 11:22.400 --> 11:28.120 So we can start to have that conversation knowing more about the health of the community and 11:28.120 --> 11:32.280 why we might consider this risky. 11:32.280 --> 11:40.400 I also have an overview of the tools that we have used, these are the Grimola tool from the 11:40.400 --> 11:49.520 Chaos community, which has arrived out of university research, more than 20 years of history 11:49.520 --> 11:56.160 already, it's been used by many different foundations and companies, it's been used in 11:56.160 --> 12:03.800 the Mozilla, Rebel Alliance Report, and it's the foundation for different project health 12:03.800 --> 12:08.560 platforms that you might have encountered already. 12:08.560 --> 12:18.560 The process really is where the key lies, that we have the espom on one side, but then 12:18.560 --> 12:27.000 to understand the community, we go out and actually look at the activity in the community 12:27.000 --> 12:29.440 and collect that. 12:29.440 --> 12:36.000 That data needs to be enriched, we need to do the calculations, before we can analyze 12:36.000 --> 12:39.680 the health and built the score. 12:39.680 --> 12:46.800 So that's the process, one of the insights that Grimola has that we haven't used in this 12:46.800 --> 12:53.880 model yet is looking at identities, because there's a lot of conversation around knowing 12:53.880 --> 12:58.680 your contributors and knowing who is contributing to open source projects. 12:58.680 --> 13:05.720 So we have a component called sorting hat built in to consolidate identities and also 13:05.720 --> 13:13.760 track organizational affiliation and other things, we haven't used it in this model yet, 13:13.760 --> 13:19.880 but it's an interesting avenue to explore further. 13:19.880 --> 13:29.000 Also when we're thinking about data privacy, GDPR, how do we manage that so that we can 13:29.000 --> 13:31.280 work with the data? 13:31.280 --> 13:38.080 If you're interested, I have this chart about the architecture, the slides are all online 13:38.080 --> 13:43.720 in the schedule, so if you want to go there, you can explore that further. 13:43.720 --> 13:50.720 Let me talk about the roadmap for Grimola, we are working on scaling right now with the 13:50.720 --> 13:57.360 use case that the having S-bombs and we're looking at tens of thousands of components 13:57.360 --> 14:01.120 at a time, rather than one community at a time. 14:01.120 --> 14:06.240 There's some advancement we need to do in the tools to scale, and that's what we're working 14:06.240 --> 14:07.800 on right now. 14:07.800 --> 14:13.240 Here's the way to get started, you can work with the open source project, or you can 14:13.240 --> 14:19.760 come talk to me, I'm happy to see about commercial support for that as well, and I want 14:19.760 --> 14:28.760 to leave you with culture action, because one of the challenges we have run into with 14:28.760 --> 14:36.040 trying to analyze the components listed in S-bombs is we can't always find where the original 14:36.040 --> 14:38.360 source came from. 14:38.360 --> 14:46.520 So if anyone building S-bombs, if you could include the origin, that would be very helpful 14:46.520 --> 14:53.800 to take the next step of looking at project health, because having only the list, looking 14:53.800 --> 15:00.400 at the components and adding that project health really helps us understand better what 15:00.400 --> 15:02.720 we are relying on. 15:02.720 --> 15:10.080 And that's what I'm leaving you with, we've done almost 60 slides in 15 minutes, so thank 15:10.080 --> 15:13.200 you for your attention. 15:13.200 --> 15:17.200 The questions together? 15:17.200 --> 15:18.200 Yes. 15:18.200 --> 15:19.200 Yes. 15:19.200 --> 15:32.680 The question is whether we can reuse the metrics from the S-bombs, health, product, 15:32.680 --> 15:40.320 project, in Prometheus, but we can use them in Prometheus. 15:40.320 --> 15:48.600 So what's the S-bombs health project, I'm not familiar with that project? 15:48.600 --> 16:02.640 So, okay, this data right here, yeah, yeah, the data is stored in a elastic database, 16:02.720 --> 16:08.880 or an open search database, we switched away from elastic, or the license, the documents, 16:08.880 --> 16:11.440 and went to with open search fork. 16:11.440 --> 16:16.160 So yeah, the data is in there, I'm not familiar with how to get into Prometheus, but 16:16.160 --> 16:18.160 I don't see why not. 16:18.160 --> 16:19.160 Yeah, thank you. 16:19.160 --> 16:20.160 Yes. 16:20.160 --> 16:32.600 It's not a question, but let's do it. 16:32.600 --> 16:33.600 Yeah. 16:33.600 --> 16:34.600 Yeah. 16:34.600 --> 16:35.600 10,000 sub-projects. 16:35.600 --> 16:41.600 And now you realize these compliance guys, so I see the component that they use in Prometheus, 16:41.600 --> 16:44.560 and I'm talking to the developers. 16:44.560 --> 16:49.240 And now we have a large number of developers teams who could tell me by heart, which components 16:49.240 --> 16:53.360 they are using, and they know the main case, so all of this could be fine. 16:53.360 --> 16:59.560 And then they are the others, the JavaScript project, the 16 levels of dependencies, how 16:59.560 --> 17:06.560 to do it there, because our people would tell me, we are not interested in this, and 17:06.560 --> 17:09.960 I can tell them, you should know your components. 17:09.960 --> 17:16.880 You have to know that because of cyber-resilience, but it still, you only need the combined 17:16.880 --> 17:17.880 stuff. 17:17.880 --> 17:22.480 So I guess most of the people sitting here in the room, they are very relevant, but what 17:22.480 --> 17:28.480 you are telling, but how do we get this more to the developer? 17:28.480 --> 17:31.720 Yeah, thank you. 17:31.720 --> 17:41.840 So for the livestream, the scenario was as the compliance person with tens of thousands 17:41.840 --> 17:48.240 of dependencies and developers, the developers are working in their day-to-day work. 17:48.240 --> 17:54.840 They know their direct dependencies, they trust the source that they are using, but then 17:54.880 --> 18:02.360 there are other situations like in Java, where we have JavaScript with lots of dependencies 18:02.360 --> 18:08.200 that the developers don't care about, and like who are you as the compliance person to 18:08.200 --> 18:10.400 make me want to care about these? 18:10.400 --> 18:11.880 I don't. 18:11.880 --> 18:20.560 How do we have that conversation to raise awareness to get more compliance that we are required 18:20.600 --> 18:25.440 to have in the future, or we should be aiming toward? 18:25.440 --> 18:30.400 So that is a very good pointer, and I see lots of hands going out. 18:30.400 --> 18:32.200 I think you were first. 18:32.200 --> 18:33.840 Yeah, you're in the green. 18:33.840 --> 18:41.240 So how do you see this enabling better automation of open SSF best practices rather than 18:41.240 --> 18:47.600 if being periodically, or maybe never again, completely non-quested? 18:47.600 --> 18:54.200 So the question is, how this can help automate like the open SSF best practices rather 18:54.200 --> 18:58.720 than being a periodic check? 18:58.720 --> 19:03.960 The open SSF best practice batch, which is fantastic, and I'm the person behind the 19:03.960 --> 19:07.680 German translation of those, so I know them very well. 19:07.680 --> 19:13.720 I love that they are a way for projects to take us to back and think, are we doing all 19:13.720 --> 19:16.400 the right things? 19:16.400 --> 19:22.840 What I'm proposing here is not adding burden on to the maintainers, it's something that 19:22.840 --> 19:31.200 companies can do at scale to look across their dependencies and start conversations. 19:31.200 --> 19:37.680 The ideal situation that I wish we would arrive at in the future is that focusing on project 19:37.680 --> 19:43.520 health leads to healthy conversations and relationships with our communities that we depend 19:43.520 --> 19:46.360 on. 19:46.360 --> 19:48.680 So it's a different focus. 19:48.680 --> 19:50.400 The next hand I saw back there. 19:50.400 --> 19:54.960 The question is, is there a lot of people that don't have to solve the voice in your 19:54.960 --> 19:59.720 experience problem, and we can do two things like the surface of this information to 19:59.720 --> 20:06.040 identify the errors, so whether we're going to do something, we show that this is bad. 20:06.040 --> 20:11.400 I don't know if she is great, can you be it? 20:11.400 --> 20:17.200 So it was a comment that we can do two things. 20:17.200 --> 20:24.240 One is, raise the awareness during the development and built phase of the software development, 20:24.240 --> 20:29.560 raise it to the developer, maybe through the CI-CD pipeline, and the second thing is to break 20:29.560 --> 20:31.560 builds. 20:31.560 --> 20:34.200 So that was the proposal. 20:34.200 --> 20:39.680 So correctly, if I want to scan some projects and get this data, I have to set up the 20:39.680 --> 20:44.920 instance, and then scan the projects and everything, so is there any effort to provide 20:44.920 --> 20:52.480 any open data on scan and like a bunch of critical projects like, for example, as a tool 20:52.480 --> 20:59.600 maintainer, it's a bit easier to integrate committee and metrics when some data is obviously, 20:59.600 --> 21:05.280 for example, the scorecard provides sometimes like data on some critical projects, so it's 21:05.280 --> 21:10.080 a bit easier to get the data through if you're going to run the scorecard to ourselves. 21:10.080 --> 21:15.440 So is there any like plan to send any open data? 21:15.440 --> 21:20.960 Yes, so the question was whether someone wants to use this approach has to stand up their 21:20.960 --> 21:27.040 own tooling, or if there is already a service where you can just ask, hey, what is the 21:27.040 --> 21:28.640 health score? 21:28.640 --> 21:30.840 And whether there's movement in that direction. 21:30.840 --> 21:36.000 And the answer is, yes, we are currently working on that to build out, I don't know if 21:36.000 --> 21:41.200 it'll be an open data set, it is something that we're working on building out an API, 21:41.200 --> 21:46.120 and I just had a conversation with Michael who had the talk right before here to see if 21:46.120 --> 21:53.080 that is building out the bigger database with more stakeholders is something we can do. 21:53.080 --> 21:55.680 Yeah, but it is something we're thinking about. 21:55.680 --> 21:56.680 Thank you. 21:56.680 --> 21:57.680 Point. 21:57.680 --> 21:58.680 Yes. 21:58.680 --> 22:08.680 We've got scorecards, we've got critical projects, 80% of the open source projects are small, 22:08.680 --> 22:10.680 one-two people, outpicks. 22:10.680 --> 22:15.120 There's a danger that what you're going to do is end up highlighting the bigger projects 22:15.120 --> 22:20.680 with me, the health, and the real small innovation was from the hobbyists that actually 22:20.680 --> 22:21.680 we rely on. 22:21.680 --> 22:22.680 Yeah. 22:22.680 --> 22:26.680 Get ignored and don't get the love and attention they want. 22:26.680 --> 22:27.680 Right. 22:27.680 --> 22:32.600 Well, I can see what you're doing, this is great to then surprise is how to be sure they don't 22:32.600 --> 22:38.000 just focus on the big projects, all the volume well funded and well resourced and we don't 22:38.000 --> 22:40.360 get ignore the little projects. 22:40.360 --> 22:44.720 Because I see that as a single maintainer, I see people doing metrics and I get a 22:44.720 --> 22:47.000 poor score because they don't need it. 22:47.000 --> 22:48.000 Right. 22:48.120 --> 22:55.760 So the question was, we already have metrics and one of the experiences that it tends 22:55.760 --> 23:01.360 to draw attention to the big projects that already have a lot of attention or funding and 23:01.360 --> 23:08.080 the small projects with just one maintainer are not getting as much attention. 23:08.080 --> 23:17.600 And what I'm thinking is, for one, this approach where we can change the parameters, the goal 23:17.680 --> 23:23.520 is to find a place where we can pay attention to because we only have limited attention, 23:23.520 --> 23:27.800 we need to filter out what it is that we pay attention to. 23:27.800 --> 23:34.280 And so looking in the long tail, trying to find where do we actually want to spend our 23:34.280 --> 23:35.600 focus on? 23:35.600 --> 23:42.520 And I think looking at the health, the smaller projects and the metrics that I've been 23:42.520 --> 23:47.320 looking, or that I was proposing here, are highlighting the smaller projects more 23:47.320 --> 23:53.560 so than the big projects, because they already have a lot of people, they already have 23:53.560 --> 23:55.440 other things going on. 23:55.440 --> 24:04.840 And so it's a matter of waiting, yes, and from our experiment, we have seen that different 24:04.920 --> 24:09.280 departments prefer different waitings. 24:09.280 --> 24:19.800 And so that is something I think as the built tools, we need to allow for that flexibility. 24:19.800 --> 24:20.800 Thank you very much. 24:20.800 --> 24:21.800 Thank you, Gary.