WEBVTT 00:00.000 --> 00:15.000 Hello everyone, my name is Marin Parich and I'm a staff engineer. 00:15.000 --> 00:16.000 Laura? 00:16.000 --> 00:17.000 All right. 00:17.000 --> 00:19.000 Hi everyone. 00:19.000 --> 00:22.000 Nice to see such a nice turnout. 00:22.000 --> 00:25.000 Yeah, my name is Marin Parich and I'm a staff engineer at I do. 00:25.000 --> 00:26.000 Social enterprise. 00:26.000 --> 00:31.000 We make a learning platform for children and low and middle income countries. 00:31.000 --> 00:32.000 But I'm not here to talk about that. 00:32.000 --> 00:35.000 I'm here to talk about property based testing. 00:35.000 --> 00:40.000 I'd like to ask you to raise your hand if you've used property based testing before. 00:40.000 --> 00:42.000 All right, a fair few. 00:42.000 --> 00:47.000 And if you use property based testing regularly in your work. 00:47.000 --> 00:49.000 All right. 00:49.000 --> 00:50.000 All right. 00:50.000 --> 00:53.000 So I can see that this talk might be useful for at least some of you, which is great. 00:54.000 --> 00:57.000 But I'm also happy to see that some of you have familiarity with it. 00:57.000 --> 01:00.000 So property based testing. 01:00.000 --> 01:04.000 I'll contrast which what I'll call example based testing. 01:04.000 --> 01:08.000 Which I think is the conventional way that we write unit tests. 01:08.000 --> 01:12.000 If we write unit tests, I mean, I hope we all write unit tests, you know. 01:12.000 --> 01:21.000 But yeah, so conventionally you set up specific scenarios, specific expected outputs and for specific inputs. 01:21.000 --> 01:24.000 And you kind of confirm every case. 01:24.000 --> 01:30.000 So here we have a test written in Python, which tests a function that reverses a string. 01:30.000 --> 01:32.000 And we have two test cases here. 01:32.000 --> 01:36.000 We cover the edge case, but including the empty string there, good job us. 01:36.000 --> 01:42.000 It can be challenging with example based testing to come up with all the edge cases. 01:42.000 --> 01:45.000 And this is one thing that property based testing helps with. 01:45.000 --> 01:47.000 Property based testing is also more fun. 01:47.000 --> 01:49.000 It's more fun to add these tests. 01:49.000 --> 01:59.000 So in property based testing, what you do is you specify properties rather than. 01:59.000 --> 02:10.000 Rather than specific scenarios that you specify general truth or invariance that you told for all inputs or that you told under certain conditions. 02:10.000 --> 02:18.000 And then the inputs for your tests are automatically generated and your tests will run hundreds or thousands of times with these inputs. 02:19.000 --> 02:25.000 And there's a little example here, again, Python with library called hypothesis. 02:25.000 --> 02:31.000 Where we are also testing reversing a string, but testing it hundreds and thousands of times. 02:31.000 --> 02:37.000 Where we're just testing that if we reverse a string twice, we get the original string back. 02:37.000 --> 02:45.000 So this is kind of a property of a reverse function that we're testing that we're exploiting here in the test that it's reversible if we invoke a twice. 02:45.000 --> 02:51.000 And that's what we can do with a property based test here. 02:51.000 --> 03:02.000 So when the property based test runs, what we see is that firstly input is generated by the testing framework. 03:02.000 --> 03:08.000 And then the property that we've set up in this case that the function is reversible is checked. 03:08.000 --> 03:18.000 And if the test passes, we just go back, we generate more inputs, we do this hundreds of times, we do this thousands of times before we satisfy and say that test is green, the test is passed. 03:18.000 --> 03:25.000 But if the test fails, there's an additional clever step that property based testing. 03:25.000 --> 03:31.000 And that makes it even more useful, which is shrinking the input. 03:31.000 --> 03:38.000 So the input will be simplified, for example, if it's a string, but for a shorter string. 03:38.000 --> 03:47.000 And an attempt to still find examples that are simpler or shorter, that still failed the test, where the property still doesn't hold. 03:47.000 --> 03:56.000 And in the end, after shrinking a number of times, the failed test you get actually will show you the last value that it was shrunk with. 03:56.000 --> 04:12.000 So that you instead of like a really long, horrible string that initially failed your test to get the shortest possible one that the system was able to find that failed your test. 04:12.000 --> 04:18.000 So it's not a little bit about input generation and property based testing. 04:18.000 --> 04:25.000 So there is some randomness involved in generating these tests at these inputs, but it's not uniform. 04:25.000 --> 04:35.000 Special edge cases are always considered, which means that we as a programmer don't have to remember them for strings that might be the empty string, it might be non-asky characters. 04:35.000 --> 04:41.000 For real numbers, that might be between tricky values, like the infinities, not a number. 04:41.000 --> 04:53.000 For a list, that might be including duplicate items in your list, because if you may say make a list of 32-bit integers at random, then at random there's value going to be any duplicates in it. 04:53.000 --> 05:01.000 Because there's so many integers, but in a property based testing framework, lists will also contain duplicates, so you can test your functions with that. 05:01.000 --> 05:06.000 And you can combine these generators to produce more complex data. 05:06.000 --> 05:13.000 Some testing frameworks call them generators, all of them arbitraries, but it's the same thing. 05:13.000 --> 05:20.000 We need to specify which inputs we want, and then the testing framework generates these. 05:20.000 --> 05:29.000 And when it fails, we also get the random seed that was used to run the tests, so that we can be sure that we can reproduce these failing results. 05:31.000 --> 05:36.000 So let's talk a little bit about properties. 05:36.000 --> 05:40.000 So we can consider properties as invariants. 05:40.000 --> 05:52.000 So what holds two across all inputs, or across the class of inputs, when we invoke a function, what doesn't change after we apply the function on a test. 05:52.000 --> 05:59.000 But anything can be a property, we can test in theory anything that we can test with an example based test, also with a property based test. 05:59.000 --> 06:05.000 We just have to come up with a particular property of our system under test. 06:05.000 --> 06:14.000 So generally, for some inputs, maybe satisfying some precondition to narrow it down to a class of inputs, we specify a predicate. 06:14.000 --> 06:30.000 And for example, a predicate might be if we're testing a function that concatenates two strings, that when we take the length of the concatenation, that should be the sum of the length of the two strings. 06:30.000 --> 06:42.000 And I'll go over some more handy properties that we can identify, because part of the trick to property based testing is finding ways to apply it. 06:42.000 --> 06:48.000 Finding properties of your systems or functions that you can test in this way. 06:48.000 --> 07:04.000 And you can take some inspiration from maths there, so a lot of these properties come from mathematics, but they apply to all sorts of things, not just mathematical constructs, but various more practical situations. 07:05.000 --> 07:15.000 So we have this two activity, which I just mentioned, this we'll be doing over the length of strings, we can also do this with any function that say transforms the strings. 07:15.000 --> 07:22.000 So if we take the upper case of two strings and we combine that, that should be the same as combining them, then taking the upper case. 07:22.000 --> 07:31.000 We have commutativity, I mean it's important if you say building a big integral library or something, that you have to check that your addition is actually commutative. 07:31.000 --> 07:38.000 And it's nice to do that with property based testing as opposed to just writing out three examples and that being satisfied. 07:38.000 --> 07:50.000 But it also comes up in other contexts, like when we have an image and we flip it once horizontally and then vertically, that should be the same as flipping it vertically and then horizontally. 07:50.000 --> 07:56.000 Or if we have some data pipeline, be that in a collection or a database or something else. 07:57.000 --> 08:04.000 If we filter and then sort or if we've sort and then filter, I would say that should again be the same. 08:04.000 --> 08:16.000 Yeah, so another property from maths again, associativity, kind of applied to concatenating strings, kind of applied to concatenating lists as well. 08:16.000 --> 08:26.000 And then some functions are invertible, like we saw, we can reverse a string, reverse a twice, we get the same thing, we flip an image twice, we get the same thing. 08:26.000 --> 08:37.000 And then some functions that have a natural counterpart. So if you're writing a serialization engine, then of course you can serialize and deserialize and confirm that you get the same data on the round trip. 08:37.000 --> 08:43.000 You can do this with encryption, decryption, you can do it with compression, decompression, if your compression is lossless. 08:43.000 --> 08:55.000 If you have some kind of editing functionality and you have an undo redo functionality, you can also apply it there, you can just perform some random actions and undo them and you should get the same state back. 08:55.000 --> 09:00.000 So all this you can state in a property based test, these are all like possible properties. 09:00.000 --> 09:14.000 Item currency of course also is a super common one in computing, the classic example would be sorting, sorting a list twice should give you the same output as sorting it just once. 09:14.000 --> 09:23.000 And maybe end of my time. I have compiled also for some popular languages list of property based testing libraries that go with them. 09:23.000 --> 09:30.000 So if you want to try it out and you see a language of choice here, then yeah, give it a try. It's fun. 09:30.000 --> 09:42.000 And I think at this point I'll have time for a couple of questions, maybe. 09:42.000 --> 09:46.000 Thank you. Thank you so much for listening. Yes. 09:46.000 --> 09:56.000 You're least sure with a lot of programming languages. We have also all the tools for generators when I'm using all the languages which are not here. 09:56.000 --> 10:04.000 So your question is, there's a lot of programming languages in the list and you're asking for a programming languages that aren't in the list. 10:04.000 --> 10:11.000 Yeah, but I have all the languages like we are here and I need to generate those. 10:11.000 --> 10:18.000 To do the in the loop, but using some all all testing in the background, you know who is doing that. 10:18.000 --> 10:25.000 I'm not sure I get the question. I'm sorry. Let's talk after. Sorry. 10:25.000 --> 10:34.000 One more question. Yes, please. 10:34.000 --> 10:39.000 Yes, in conjunction with example based testing. Yes, in conjunction with. It's always good to write out a few example based tests. 10:39.000 --> 10:45.000 There can be nice food documentation. Like you can see how exactly you want your API to work. 10:45.000 --> 10:51.000 And there are certain things that aren't handling the cover property based tests, but there's some things that property based tests do much better. 10:51.000 --> 11:01.000 And they can also use to document these properties that your system might have. So they pair nicely with each other. 11:01.000 --> 11:06.000 All right then. Thank you.