WEBVTT 00:00.000 --> 00:18.000 I am a senior researcher at Yale Intering Institute and a co-director for OLS, open life science, 00:18.000 --> 00:20.000 depth stocked about it. 00:20.000 --> 00:26.000 But today, also, we're working after 5 PM, so I'm not representing any of my affiliation. 00:26.000 --> 00:28.000 I'm representing myself. 00:28.000 --> 00:30.000 I am from India. 00:30.000 --> 00:36.000 I'm from North part of India, northeast from a small town called Ranchi. 00:36.000 --> 00:38.000 It's small town for Indian standards. 00:38.000 --> 00:42.000 We have about 1 million people. 00:42.000 --> 00:46.000 And I moved to Germany. 00:46.000 --> 00:50.000 I moved from North India to South India very early on because I wanted to study 00:50.000 --> 00:55.000 computation and I wanted to do something which was emerging and wasn't really available everywhere. 00:55.000 --> 00:57.000 So I went to Bangalore. 00:57.000 --> 00:59.000 I'm sure you might have heard about that. 00:59.000 --> 01:05.000 And I was very convinced that I wanted to move abroad and learn about 01:05.000 --> 01:09.000 mathematics, change my field, and I moved to Germany. 01:09.000 --> 01:11.000 And that's where I spent about a decade. 01:11.000 --> 01:15.000 And after living there and after really reflecting back on my life, 01:15.000 --> 01:23.000 I really think that my attraction towards open science, open source, was really driven by this 01:23.000 --> 01:29.000 perspective, this identity being in a country that is not mine, trying to fit in, 01:29.000 --> 01:33.000 and open source of science felt like something that was enabling access. 01:33.000 --> 01:37.000 So being in Germany, I had a lot of access to different resources, which I didn't have 01:37.000 --> 01:39.000 back in India and my colleagues still don't have. 01:39.000 --> 01:43.000 So I went to do my PhD. 01:43.000 --> 01:45.000 That's where I got a lot more involved with open source. 01:45.000 --> 01:49.000 And I was very convinced that I'm going to move towards community management, community building. 01:49.000 --> 01:51.000 And that's where I'm working now. 01:51.000 --> 01:55.000 That has become my profession, and I'm really proud to even build next generation 01:55.000 --> 01:57.000 of community managers who are part of my team. 01:57.000 --> 02:03.000 So I am a co-founder for OLS, which came out of Mozilla. 02:03.000 --> 02:05.000 It's a happy game. 02:05.000 --> 02:13.000 We were part of cohort, which were incubated when Mozilla opened leadership could not be funded anymore. 02:13.000 --> 02:17.000 And a lot of people like us who wanted to make open science open source accessible, 02:17.000 --> 02:21.000 went on to build something that we thought our community needed. 02:21.000 --> 02:26.000 I'm also a lead for the touring way, and a lot of the contributors are in the room, 02:26.000 --> 02:29.000 which is a community like guide for data science best practices. 02:29.000 --> 02:32.000 So there's a book, but there are people who are working in this book. 02:32.000 --> 02:39.000 And the process of building has really generated momentum on the kind of culture we want. 02:39.000 --> 02:44.000 Kind of kind, equitable practices that we want to promote in open source. 02:44.000 --> 02:48.000 So I've been involved in quite a lot of open science open source projects, 02:48.000 --> 02:52.000 and you can call me a quite compulsive open science community facilitator. 02:52.000 --> 02:56.000 So sometimes I do things that I don't know why I do them. 02:56.000 --> 03:00.000 So I came across do no harm project few years ago. 03:00.000 --> 03:04.000 The do no harm project talks about how sometimes, 03:04.000 --> 03:10.000 thoughtless data science can lead to really outsized damage on communities, 03:10.000 --> 03:13.000 particularly from marginalized community. 03:13.000 --> 03:20.000 So they built a series of guides which invites analyst researchers, 03:20.000 --> 03:26.000 artists and activists to apply equity diversity and inclusion lens 03:26.000 --> 03:29.000 to encourage the hardful way of doing data science. 03:29.000 --> 03:32.000 In a way that it does not do any harm. 03:32.000 --> 03:35.000 So of course it was very close to what I was working on. 03:35.000 --> 03:39.000 I got very excited, I heard about do no harm in different context, 03:39.000 --> 03:42.000 and this time it was first time for data science. 03:42.000 --> 03:46.000 So all the projects that I found, all the reports that I found, 03:46.000 --> 03:48.000 sorry, see this in there. 03:48.000 --> 03:52.000 We're around how to make equitable data narrative, 03:52.000 --> 03:56.000 how to improve perspective, how to do better visualization, 03:56.000 --> 03:58.000 and they are really, really amazing. 03:58.000 --> 04:02.000 But across those reports, I did not see any mention of open source open science, 04:02.000 --> 04:04.000 despite it was a lot about technology and data. 04:04.000 --> 04:07.000 So really, maybe wonder, what is it signaling? 04:07.000 --> 04:11.000 Is it signaling that open source community does not have to care about 04:11.000 --> 04:15.000 do no harm principle, or was it signaling that we're doing really well? 04:15.000 --> 04:19.000 Because we're all lovely community people and like each other. 04:19.000 --> 04:21.000 So we are never even thinking about harm. 04:21.000 --> 04:26.000 So I got very, very curious, and I really had to solve that problem. 04:26.000 --> 04:31.000 So I reached out to Urban Institute and proposed this idea 04:31.000 --> 04:36.000 and this time do no harm in the context of open source and open science. 04:36.000 --> 04:38.000 Yeah, that's a highlight. 04:38.000 --> 04:41.000 So this report is published now that you can read, 04:41.000 --> 04:45.000 but I'll talk you through so you don't have to read. 04:45.000 --> 04:49.000 So the first part of this research was to understand 04:49.000 --> 04:53.000 is open practice at all compatible with do no harm. 04:53.000 --> 04:55.000 Is it the right route to go towards, 04:55.000 --> 04:58.000 or am I looking at do no harm from a very community perspective 04:58.000 --> 05:01.000 that may not apply in open science? 05:01.000 --> 05:05.000 So do no harm has its origin in apocratic oath. 05:05.000 --> 05:08.000 So anybody who becomes a doctor has to take this oath 05:08.000 --> 05:12.000 of not harming their community or not mistreating their patients 05:12.000 --> 05:14.000 or not doing something that can harm them. 05:14.000 --> 05:18.000 So this is almost like a loyalty towards their profession. 05:18.000 --> 05:20.000 And this does not have been in research. 05:20.000 --> 05:23.000 It does not happen in research suffering engineering. 05:23.000 --> 05:25.000 It does not happen in open source. 05:25.000 --> 05:28.000 So there's a lot of mutual trust. 05:28.000 --> 05:31.000 We believe that we are all going to behave properly, 05:31.000 --> 05:32.000 all we find. 05:32.000 --> 05:35.000 So we don't need to tell anything. 05:35.000 --> 05:38.000 But do no harm was then adopted in humanitarian aid work 05:38.000 --> 05:43.000 because humanitarian aid works were often supplied in places with wars 05:43.000 --> 05:44.000 and conflicts. 05:44.000 --> 05:46.000 A lot of people's lives were in danger. 05:46.000 --> 05:52.000 So they started to use that as a way to promote similar ideology. 05:52.000 --> 05:55.000 So there are many different definitions. 05:55.000 --> 05:59.000 And a lot of time do no harm can be understood differently 05:59.000 --> 06:00.000 based on where people are. 06:00.000 --> 06:04.000 But this is one of the things, one of the definitions that I really like 06:04.000 --> 06:09.000 which is about stepping back, taking a look at what we are doing 06:09.000 --> 06:13.000 and identifying is there a place where my work might have been causing harm 06:13.000 --> 06:16.000 to the local community, to their environment 06:16.000 --> 06:22.000 or the fabric of society and economy. 06:22.000 --> 06:24.000 So here's a little bit history. 06:24.000 --> 06:28.000 As I was saying, it started in apocratic oath quite early on. 06:28.000 --> 06:30.000 I don't even know the dates. 06:30.000 --> 06:33.000 And in 90s it was then expanded into humanitarian aid work. 06:33.000 --> 06:38.000 CDA was looking at do no harm to understand in conflict 06:38.000 --> 06:40.000 and war zone. 06:40.000 --> 06:44.000 And then around that time, a lot of people should objection 06:44.000 --> 06:46.000 for using do no harm out of context. 06:46.000 --> 06:50.000 But luckily in 2014, a lot of disciplines including research, 06:50.000 --> 06:56.000 academic research where do no harm was being applied to. 06:56.000 --> 07:00.000 But one thing to say, do no harm really can not be achieved. 07:00.000 --> 07:04.000 Even a doctor cannot think about doing no harm. 07:04.000 --> 07:07.000 Even the act of taking a bit of blood from your body 07:07.000 --> 07:09.000 causes a little bit of harm. 07:09.000 --> 07:12.000 So it's understood that do no harm literally does not mean do no harm. 07:12.000 --> 07:16.000 It's about doing not no significant harm or no intentional harm. 07:16.000 --> 07:18.000 So that's where I'll be also looking at. 07:18.000 --> 07:23.000 Can we think about intentional or significant harm rather than no harm at all? 07:23.000 --> 07:28.000 Especially as we know, data technology will always harm somebody. 07:28.000 --> 07:32.000 So what the good news was that looking at literature and publications, 07:32.000 --> 07:36.000 I could find that there were a lot of alignment between do no harm 07:36.000 --> 07:41.000 area of scope and what we were doing in open science. 07:41.000 --> 07:46.000 So there are a few areas where there are known harms of data and technology. 07:46.000 --> 07:48.000 So abuses of human rights. 07:48.000 --> 07:51.000 We are hearing a lot about it, conflicts and wars 07:51.000 --> 07:54.000 where data is used to oppress people or there. 07:54.000 --> 07:59.000 Now AI is being applied in making war in a different scale. 07:59.000 --> 08:04.000 Environmental impact of the work people do and the production of abusive products. 08:04.000 --> 08:08.000 So open source community does not have a say that if we release a software, 08:08.000 --> 08:11.000 nobody can use them for any of those purpose. 08:11.000 --> 08:17.000 We do have licenses and we do want people to apply our code of conduct. 08:17.000 --> 08:22.000 But there's no way we can monitor that our open source tools are not applied to any of these. 08:22.000 --> 08:30.000 And it's very scary thing to think about that some of the work we might have published may have gone into building something that we're not proud of. 08:30.000 --> 08:34.000 But there are way more other things that we don't talk about, 08:34.000 --> 08:37.000 which could be as simple as job displacement. 08:37.000 --> 08:44.000 The automation is causing lots of job displacement without any commitment from organizations to re-skilled their worker. 08:44.000 --> 08:49.000 There are things like threats of taking data use on for incentives. 08:49.000 --> 08:54.000 I'm going to let you read and I actually ran out of space so I didn't even look for more things. 08:54.000 --> 08:59.000 There are a lot of harms that we are not looking at on our day-to-day basis for whatever reason. 08:59.000 --> 09:02.000 We're busy, we're writing grants all the time. 09:02.000 --> 09:04.000 We have a PhD to finish. 09:04.000 --> 09:07.000 A lot of things that we think are our life goals. 09:07.000 --> 09:13.000 But the act of doing a harm is to really then step back and look at our life goal in the context of society. 09:13.000 --> 09:21.000 So which may be more motivated to see okay, these things are where we can think about doing a harm principle. 09:21.000 --> 09:24.000 So there are three main factors of doing a harm. 09:24.000 --> 09:26.000 One is intervention. 09:26.000 --> 09:29.000 So let's say an aid work that we are supplying. 09:29.000 --> 09:35.000 Environment, the location where this aid work is being supplied to, and beneficiary. 09:35.000 --> 09:38.000 This aid work should benefit somebody. 09:38.000 --> 09:44.000 If you look at it in the open science perspective or open source or any kind of open practice, 09:44.000 --> 09:47.000 you can think about each practice as its own intervention. 09:47.000 --> 09:53.000 So open practice, for example, you have different kind of open source, open data, open access, 09:53.000 --> 10:00.000 open education, citizen science, think about all these different kinds of practice as one of the interventions. 10:00.000 --> 10:07.000 The environment is local environment, natural environment, as well as the workplace, the impact of our working condition. 10:07.000 --> 10:14.000 And the beneficiary should include always include diverse researchers as well as social actors. 10:14.000 --> 10:20.000 So we have been talking about working with communities where our communities are also really confined to the area that we are working in. 10:20.000 --> 10:29.000 And not so much from outside world where let's say an aid technology will be applied to impact local community. 10:29.000 --> 10:35.000 So there's another thing to note that open science itself is highly relative, highly contextual, 10:35.000 --> 10:38.000 depending on people where they are. 10:38.000 --> 10:44.000 It is affected by local realities of who is using this intervention and for what purpose. 10:44.000 --> 10:48.000 As well as it also impacts, it gets impacted by the intervention, right? 10:48.000 --> 10:56.000 We heard really good talk from Dan talking about funding and talking about people's need and what their focuses are. 10:56.000 --> 11:01.000 But also we have governance and community and policy and legality. 11:01.000 --> 11:11.000 A lot of different things that are intervention on open science which further impacts how we are applying that into a work. 11:11.000 --> 11:14.000 So I'm not the first person who was thinking about it. 11:14.000 --> 11:23.000 I was listening to sustained podcast and someone talked about to know harm license and that's how I found out that there was indeed a do know harm license. 11:23.000 --> 11:26.000 However, that license never became legalized. 11:26.000 --> 11:33.000 There's a group of people who are working on ethical source licensing and trying to think about how can we enforce some of these, 11:33.000 --> 11:40.000 enforce against these misuses of our open source software. 11:40.000 --> 11:43.000 So part two, so I'm going to share a bit of insight. 11:43.000 --> 11:51.000 So we talked about that there are compatibility we can think about intervention environment and beneficiaries in the context of open science. 11:51.000 --> 11:57.000 So there are four categories that I would invite you to think about do know harm. 11:57.000 --> 12:01.000 So intervention in the context of rights of people. 12:01.000 --> 12:05.000 So all actors of open source communities and their human rights to science. 12:05.000 --> 12:10.000 The second is functioning of the community, governance participation and recognition of people. 12:10.000 --> 12:12.000 The third is environmental context. 12:12.000 --> 12:19.000 So we explained about where they are as well as the environmental impact and the local economy of researcher. 12:19.000 --> 12:21.000 There's something I really care about. 12:21.000 --> 12:24.000 It's not just enough to say, here you go, I'm going to give you authorship. 12:24.000 --> 12:27.000 And I'm going to build a product that's going to make me very, very rich. 12:27.000 --> 12:31.000 We have two wealth wise, share our wealth with people. 12:31.000 --> 12:42.000 Especially my marginalized communities who are often just a participant of technology building rather than the people who get to set the agenda. 12:42.000 --> 12:47.000 And of course the framework is not useful if we are not thinking about practical solutions. 12:47.000 --> 13:01.000 So this framework should allow people to have conversation around these four areas and identify infrastructure implementation or practices and behavior change that can help them mitigate some of these harms. 13:01.000 --> 13:11.000 So what this could look like in practice and this is a very experimental slide because I want your help in actually looking at if this is a practical thing for us. 13:11.000 --> 13:23.000 So example word are we paying for the hidden cost of genuine and I don't know if you like AI but I thought let's talk about AI for one more time. 13:23.000 --> 13:33.000 Chad Chippete came out in 2022 and it was really bad and we really complained about it but then it became really fun and people were very much into it. 13:33.000 --> 13:49.000 So this is a reddit thread where someone's like, I love Chad Chippete much better than Google and then a lot of people are sharing, oh yeah, I asked Chad Chippete for what cocktails should I make or I just don't even use Google anymore. 13:49.000 --> 14:01.000 So is it fun right it's fun is it fun and the fact is actually there are quite a really high cost for who pays the price for these fun act. 14:01.000 --> 14:13.000 So when we put our query in Google we are using energy equivalent to running a 60 watt bulb for 17 seconds. 14:13.000 --> 14:19.000 And we do a query search in Chad Chippete it's about three minutes of light. 14:19.000 --> 14:37.000 So I did some unnecessary calculation despite that calculation somewhere exists. So there are 100 million queries per day actually more than that which can result into 226 giga watt per year which can have a five minutes left. 14:37.000 --> 14:43.000 So these two entries like Belgium can have an entire of electricity with that. 14:43.000 --> 14:51.000 Chad Chippete tree training alone emitted 550 metric ton of carbon and carbon emission of 100 cars in its entire life cycle. 14:51.000 --> 14:59.000 I didn't even go into like really scary data just wanted to look at the environmental impact in terms of energy. 14:59.000 --> 15:09.000 But there's also this really interesting paper on the Thursday AI. So of course these data centers require cooling cooling cannot happen with just normal water. 15:09.000 --> 15:17.000 It has to be very very clean water and the amount of water that goes into cooling these are really interesting. 15:17.000 --> 15:23.000 They did a couple of times during this presentation preparation, music a couple of Chad Chippete things. 15:23.000 --> 15:39.000 And I had to remind myself of going to go to a bit of water every time because every time we're doing it we are actually wasting some water that a lot of countries don't have access to. 15:39.000 --> 15:45.000 So it's a little impact of environmental impact of AI in general speaking. 15:45.000 --> 15:53.000 And we're going to skip that for our purpose because we're talking about we've already covered that right. 15:53.000 --> 16:01.000 So there are discrimination bias, there are job displacement, there are people who don't have anybody they can hold accountable. 16:01.000 --> 16:11.000 But then we are also talking about carbon emission, we're talking about the water and carbon footprint and also the evased that affects people in the global south. 16:11.000 --> 16:23.000 Mostly because all the evas are exported to those countries and there are manual worker who are working in really bad condition and their health and danger, their environmental and danger. 16:23.000 --> 16:29.000 So Jen AI is amazing, it's powerful and it's no way that we're going to ever stop seeing it and we should not. 16:29.000 --> 16:34.000 But we have to think about what is the optimum way to build these kinds of sources. 16:34.000 --> 16:42.000 So open source committee will say oh we should create open source great we actually do have one and there are lots of debates I'm not going to mention the name. 16:42.000 --> 16:48.000 So open source is a good alternative to chatchipity open source AI, Jen AI. 16:48.000 --> 16:54.000 But if we can think about the environmental costs we can think about the community cost. 16:54.000 --> 17:03.000 We might want to think about Jen AI in a different way and I don't know, I'm not a Jen AI expert, I'm just a user, but I think we can do better. 17:03.000 --> 17:11.000 So this is where I need your help, this is going to be the part three I want to understand, can we think about do no harm just for your sake like would I. 17:11.000 --> 17:24.000 What where I talk about do no harm in the context of the word that you're doing, do you think it's relevant should we have this conversation more than just a paper that I've published or should it not exist because it's unnecessary for this community. 17:24.000 --> 17:40.000 So we have a buff tomorrow, birds of feather which would be open for them, I don't need to give this talk again, you can ring your ideas, you can tell me how wrong I am or you can tell me all the things that I did not treat or you can tell me that this is practical and we all should be caring about it. 17:40.000 --> 18:01.000 The paper is published so you can read it, it can connect with me, I am an all social media, very less active, but with that I'll stop and I'm going to just tank the brilliant team member and all the wonderful people who are in this room and thank you all for staying here, I didn't think this room will have enough people, so I am so glad that you heard, thank you so much. 18:10.000 --> 18:15.000 Thank you all for your questions. 18:15.000 --> 18:17.000 Yes, then. 18:17.000 --> 18:32.000 Just back on the beginning of the do no harm idea, I guess I was thinking about the distribution of the reports that tried to balance the benefit of doing something virtual, possible harm. 18:32.000 --> 18:42.000 Yeah, yeah, I think the main reference I use for my research, yes, sorry. 18:42.000 --> 18:58.000 So right now I was talking about the do no harm from perspective that looking at our work and thinking if it can cause harm and I think that you're asking if we can look at if this work is causing more harm than benefit. 18:58.000 --> 19:13.000 So doing some sort of, I'm not saying he's saying it I'm rephrasing can we do some cost analysis for understanding if our work is worth pursuing because it's just going to harm people so what's what should it do. 19:13.000 --> 19:25.000 And I think that it's a very good angle and wanted to just mention that the reference that I'm using for my primary references, it says that every organization and groups should have the ability to define their own definition of do no harm. 19:25.000 --> 19:38.000 Do no harm is just a terminology and they know their community better. So I completely think that there are multiple directions we can go depending on what kind of work we're doing. Yes, please. 19:38.000 --> 19:48.000 So there's a concept of evolution for the architecture which uses different functions because there's always play ups between things. 19:48.000 --> 19:57.000 Have you talked about that work? So do less harm, but do it in your own context? Why do you find it in this function? 19:57.000 --> 19:59.000 Yeah, of course it's hard. 20:00.000 --> 20:13.000 Yeah, that's, you have beautiful minds. So I'm going to repeat that question. Thank you, Jim, for being the access warrior in this room. 20:13.000 --> 20:26.000 So the question is very similar to what Dan is asking, but in this case, there is already some sort of architecture for understanding. 20:26.000 --> 20:35.000 Yeah, so it can we do some modeling for what do we do that causes harm and how do we optimize for that? 20:36.000 --> 20:51.000 I had never thought about that and I think when you were talking, I thought, it reminded me of are we talking about things like offsetting the carbon cost, right? Like when we buy purchase of light, it also says pay for this to offset the carbon cost. 20:51.000 --> 21:06.000 I think that's an interesting interesting idea. It sounds like if there is something already in place adopting just feels very open source. So I'd be very keen to have that conversation. 21:06.000 --> 21:12.000 Oh, yes, I don't know who was first, but I was sitting next to you, so please do. Go ahead. 21:12.000 --> 21:26.000 So we are building a monoculture of Anglo-Sense Creek communication to the world. I'm thinking for example, and a senior, and two years ago, do you know? 21:26.000 --> 21:35.000 And who's knowledge, and she is showing that there's another world beyond the world of people with different languages, different cultures. 21:35.000 --> 21:42.000 And every time we talk in English on this, they're actually causing harm to them. 21:43.000 --> 21:49.000 So when the Nashiva also wrote a lot about monoculture of mine, so I'm going to repeat the question. Yes. 21:49.000 --> 21:57.000 So there are a lot of, so there are a lot of Western centric ways we communicate about different framework. 21:57.000 --> 22:10.000 And when we talk about the framework that's built in Western places, we sometimes actually by the act of talking about it in the language, which is not people's own language, we are already perpetuating harm. 22:10.000 --> 22:23.000 So I think it's a bit about English hegemony, how English is a normal language, and I have to say, like, very privileged to work with communities where people are doing translation and localization. 22:23.000 --> 22:34.000 So they can understand, it's not just about text to text translation, but cultural integration in the text that we are building. So it's relevant for that particular communities. 22:34.000 --> 22:44.000 So I think my response to that would be, if we have a room full of just same people with same idea, we are agreeing on everything. 22:44.000 --> 22:53.000 That's a sign of harmful behavior, because we are not identifying different perspective that can challenge that notion. 22:53.000 --> 23:04.000 Yeah, I think diversity is the answer, and I hope we all can continue to fight for EDI words and EDI act to remain as an important part of data science and open source. 23:04.000 --> 23:13.000 And even in places where we have the conversation around the next framework or next textonomy or next standard. So I completely agree with you on that. 23:13.000 --> 23:16.000 The one last question. 23:16.000 --> 23:25.000 Yeah, I think for you. And the one thing is that exists like a group of people that could be like an extreme legalist of rule of art. 23:25.000 --> 23:35.000 That's with a like a locate for actually forbidding by low some technologies, because they could be used in harmful ways. 23:35.000 --> 23:41.000 My personal opinion would be that the technology is mutual and that it should avoid it. 23:41.000 --> 23:42.000 Right. 23:42.000 --> 23:45.000 Just one of these efforts would be a really small session. 23:45.000 --> 23:46.000 Yeah. 23:46.000 --> 23:49.000 So I'm going to repeat the question. 23:49.000 --> 23:57.000 I remembered. So can can do no harm be misused? 23:57.000 --> 24:10.000 I think it's the meta question. Can do no harm be harmful for some context where people with extreme legal ideas can try to game this in some ways. 24:10.000 --> 24:19.000 And one statement I want to repeat is that technologies neutral and I want to say technology is not neutral at all. 24:19.000 --> 24:31.000 And the reason why I feel comfortable with do no harm because it has proven track record and humanitarian aid work and medical context where they are not legal requirements. 24:31.000 --> 24:39.000 They are commitment from the professional towards the discipline that they care about because they understand the impact on individual and people's life. 24:39.000 --> 24:51.000 Something we as a community understand how our action may have those kinds of consequences, because a lot of time we just work on a get up page right like who's going to be harmed by a get up page. 24:51.000 --> 25:06.000 So honestly, I shouldn't be caring about in those context do no harm, but I think we need to step back just the act of stepping back and saying is it just about get up page or there's something that we have missed or that was just an example. 25:06.000 --> 25:11.000 But I think people can misuse whatever they want. 25:11.000 --> 25:24.000 And if we have critical mass who cares about across and really operationalize it, I just think open source just signed up for misuse. 25:24.000 --> 25:31.000 When they say I'm going to just publish it in MIT and ask you just to attribute me and do whatever you want or you know take it in commercial. 25:31.000 --> 25:51.000 So I think we know that our work can be misused. In fact, the do no harm license was never legalized because people just couldn't agree on you know can you restrict someone from not using it for technology that you don't personally as a political person think about it's ethical but the local community thinks it's ethical. 25:51.000 --> 25:59.000 So I think it's an interesting debate and I think there's a lot of things that I don't know some very open to have all of those conversations tomorrow. 25:59.000 --> 26:01.000 On those ways words, thank you very much.