WEBVTT 00:00.000 --> 00:02.000 You 00:30.000 --> 00:32.000 You 01:00.000 --> 01:02.000 You 01:30.000 --> 01:32.000 You 02:00.000 --> 02:02.000 You 02:30.000 --> 02:32.000 You 03:00.000 --> 03:02.000 Watch 03:06.000 --> 03:09.000 Curric behöver 03:11.000 --> 03:14.000 And then 03:14.000 --> 03:16.000 Go get this 03:16.000 --> 03:20.000 And there 03:20.000 --> 03:24.000 And then 03:24.000 --> 03:27.200 The 03:27.200 --> 03:36.800 Hi everyone, my name is Manor and I'm speaking from Ghana and I took this going to be on 03:36.800 --> 03:44.200 MPF, but then it's here, this bucket is filled up and the foundation has it's in my 03:44.200 --> 03:55.200 handouts, we want to upgrade the MPF to match it so we can stop this in peer, we call it 03:55.200 --> 04:04.880 action. As me as a peer, there's only a thing, so and in developments we came across something 04:04.880 --> 04:10.000 very interesting about the performance, I initially wanted to leave that talk on the performance 04:10.000 --> 04:14.280 in the cano, but I had to come and give it, I've come with a course on the game, just 04:14.280 --> 04:19.480 when I discovered that there is a space and two, I started to give a very short talk on it 04:19.480 --> 04:29.280 and so for me, I started my name is Manor and I recently sent a couple of songs, I joined 04:29.280 --> 04:37.720 in the SPSD, I think a few months ago and I joined the coming work on an alternate Q&A 04:37.720 --> 04:45.520 year for 23 years old Q&A in MPF and it's almost a complete project and it's still in my 04:45.520 --> 04:53.320 review and yeah so the current amount on the email, join this build the org and my free 04:53.320 --> 05:00.560 base language is at C, C++ and D, so the MPF is the primary package with our front end 05:00.560 --> 05:10.560 with a good stem, support IPP-4 and IPP-6 and Lyaphyr protocol, CPE, asset of features, network 05:10.560 --> 05:16.160 addition, listen tables, almost like modern packages without you how you can build 05:16.160 --> 05:23.560 kind of content with MPF and you are good to go, the MPF has a super high efficiency 05:23.560 --> 05:32.120 I cannot very high performance can know it, which is designed to actually very skill very well 05:32.120 --> 05:45.320 so the design for both asset and MPF was actually developed by our mind to actually take 05:45.400 --> 05:55.440 advantage of multiple core CPUs, so if you actually used it before you realize that it's very 05:55.440 --> 06:01.080 efficient, it's very efficient and it's the first time I'm handled large with your clients 06:01.080 --> 06:06.920 and but in this time I've been moved, the SPSD is good, but it's a very good boost 06:07.320 --> 06:16.600 and it's very easy to understand and very interesting one, so the example of the MPF is heavily 06:16.600 --> 06:22.200 and gives the design is heavily dependent on the MP library, the BSM library and that 06:22.200 --> 06:29.960 it passes rules and components of the configuration into name value in place, under global 06:29.960 --> 06:37.960 ambient structure and I'm sure most of us will be familiar with them is library them in the 06:37.960 --> 06:50.280 structure and yes, so due to the context of the ambient structure, if we pass if you want 06:50.360 --> 06:54.040 to pass the data to the cannot, it's not really at the same time because if you're living 06:55.080 --> 07:09.560 copy, data, that's held by the MPF, it's held by MPF, you can even go and actually get the data 07:09.560 --> 07:17.000 you're copying, using our space addresses, which would also need some attention because 07:17.000 --> 07:23.640 there's just this addresses won't be valid in the cannot, so there's a need to serialize the 07:24.680 --> 07:32.520 name value pass to a binary buffer and then some of the end is binary buffer when copying 07:32.520 --> 07:40.120 in into the cannot, it's copied with elements, you cannot copy more than so you gave this data 07:40.200 --> 07:47.720 serializes buffer, it's serialized, it's more than four megabytes, it feels, you cannot copy 07:49.000 --> 07:54.040 and to the one in the line, it impacts the data buffer to the available data structure to be 07:54.040 --> 08:01.480 possessed in the cannot, so above you of the design as I said, the original name value has a very good 08:02.440 --> 08:10.760 structure, it makes the course very easy to read and understand and maintain, because the human 08:10.760 --> 08:20.120 I am really very well, if references to identify as references to unknowns, I very so much includes 08:20.120 --> 08:27.080 proximity, anything we don't know as humans, if the argument is so close to what it is, it's 08:27.400 --> 08:35.640 actually understand, that when they are far from us, so if you pass your rules like pass rule, 08:35.640 --> 08:39.400 you know the okay action to be taken, it's passing, trying to trace what it says, you actually 08:39.400 --> 08:43.640 see the right foot of you, and it's supposed to be going to different files to actually go and 08:43.640 --> 08:48.680 trace the differences of them, you actually see the right foot of you passing them and then 08:49.560 --> 08:59.560 then the advantage is that the performance overhead costs in the design copy of the data, 08:59.560 --> 09:05.640 because of the need to see what I say, you actually try to convert them to binary buffer before 09:05.640 --> 09:11.240 these sent to the cannot, so what to be super destocked, and my actually came across a link, 09:11.960 --> 09:22.920 I know Twitch, that was talking about the use of an MPF to watch a link, and to check 09:22.920 --> 09:31.480 photo for an IP table, because of 32,000 entries of IPs and a field, so I alluded to this, and 09:31.480 --> 09:39.080 when I alluded to it, I realized that it was due to the limits of the data buffer, so 09:40.040 --> 09:47.480 obviously the 52 IP tables were compacted into the data buffer, that was more than forming 09:47.480 --> 09:53.560 a byte, and so you get a nice error, and then when I was trying to do this test, I also 09:53.560 --> 09:58.920 realized that when I was new, I realized that a large IP is also taken out with you by like 09:58.920 --> 10:06.120 64 seconds to complete to complete, and that is very long, and that's the tendency, so 10:06.200 --> 10:13.240 that's the reason why I tried to keep this talk, and I tried to create awareness for that, 10:14.040 --> 10:20.280 so that course of the configuration through this for IP tables, if you run your 10:20.280 --> 10:27.560 configuration, for just 52 IPs and 52 IPs and a simple default, you are getting your 10:27.560 --> 10:33.960 it to be a error, your argument is too long, and definitely if you hear something, this is just 10:33.960 --> 10:39.480 good to come to you, that maybe there's a confusion buffer, so I changed the bytes, that will be 10:39.480 --> 10:44.520 copied, and I realized that was five megabytes, and so according to the design structure, 10:44.520 --> 10:52.040 the design philosophy of MPF, you cannot copy buffer size of more than forming a brightness, they 10:52.120 --> 11:04.920 cannot, but considering scalability is it, is it a very good approach considering if we 11:04.920 --> 11:11.000 want MPF to subscribe in research environments now, if you're going through such a 11:11.000 --> 11:18.520 parameter, people are feeding periods and clays of data, and so if we limit MPF, we allow 11:18.520 --> 11:27.800 we need to have this power, we allow MPF to reach as far as it can, because this gravity, I think 11:27.800 --> 11:35.720 we should find an dynamic way to actually load the data, not necessarily put 11:36.680 --> 11:43.160 elements to it, because I understand they need to actually minimize the memory usage in the 11:43.160 --> 11:49.560 cano, and it's always sensitive, but if we are ready to let MPF expand the research environment, 11:49.560 --> 11:56.120 and I think you have to do something about it, so I think it's a very large real time, so 11:56.120 --> 12:00.040 for a lot of our business, I think when it came, I'm so quick to blame the cano and everything like 12:01.000 --> 12:05.080 I think the cano is the cano, so I did a profile on the cano, I tell you to choose the cano 12:05.080 --> 12:12.760 and some Cisco's, and some entry and return groups, and I produce a distribution for the 12:12.760 --> 12:23.640 ISOC tiers for ISOC tiers, and I realize that the entry when the ISOC is called, the data is being 12:23.640 --> 12:29.960 the transition is being done to the cano, it's taking less than two seconds to complete, 12:29.960 --> 12:36.440 despite the huge amount of data being copied, it was taking less than two seconds to complete, so 12:37.880 --> 12:44.920 I even provided a slim graph for that scenario, that's no, we should pay attention to the ISOC space, 12:45.960 --> 12:52.280 and so I tried to also choose the ISOC space, and choose the ISOC space, then some few functions, 12:52.280 --> 12:58.280 I used the D-tracing, and it was able to cut us a few functions, so I decided to use the 12:58.280 --> 13:06.440 traditional print that method to put some print-off statements at entry and return points, 13:06.440 --> 13:12.680 function that, or into the function tree to, before I get to the ISOC system code, which sends 13:12.680 --> 13:20.680 the data into the profile, and to my observation, I realized that when the data is being 13:20.680 --> 13:28.280 packed or serialized into the profile, then to the serialized profile, it takes originally about 62 seconds, 13:29.320 --> 13:37.800 and I realized that 62 seconds was very long, but it could be happening for that to happen, 13:37.800 --> 13:46.680 because I had a start in 52,000 copies, it's very huge, but 62 seconds is intense, that's so long, 13:46.680 --> 13:52.920 so I did my own testing, I realized that I needed a new tool, just then I peed, and I thought 13:52.920 --> 13:57.560 it to print the buffer to see what was actually going on, and so I totally see the size of the buffer. 13:57.560 --> 14:03.960 So for this, then I peed, I realized that we were getting 2,000 bytes of buffer, 14:04.600 --> 14:09.880 which was the canon, and I printed what was in the canon, so it's garbage, 14:09.880 --> 14:17.080 values, and I lost to, because the size of most of our entries are very huge and bigger than the 14:17.080 --> 14:25.080 data content, so it makes sense for me, but I think, because I'm smiling, it's because if you look at 14:25.080 --> 14:31.080 the buffer size, the canon is so garbage, it's so much, a little garbage, but it's, so I thought 14:31.080 --> 14:39.560 to give a simple, to show a simple way that our data is being plugged into the buffer, and 14:41.160 --> 14:49.160 there is a simple and repeat endless table for a single, the IP, a simple, the single IP, 14:49.160 --> 14:56.680 and I decided to actually print it to show how it is being done, and if we are loading the, if we, 14:57.640 --> 15:02.360 any time we are building the configuration, it starts with a very fast root MVP, 15:02.920 --> 15:07.720 which is the version of the MPF, and the name, the version and the value of the version, 15:07.720 --> 15:14.360 and then, so for a single IP, it also creates a new table, an MVP for the table, the IP, 15:14.360 --> 15:23.080 and it's value. So, and then it's then creates the types for the table configuration, 15:23.800 --> 15:29.480 the name, the ID, the type, and it's value, it's given, and these are decided to enable this 15:29.480 --> 15:36.520 app string, and this app number, based on the type, and then the IP address has been added as 15:36.520 --> 15:42.600 a bytes, bytes, I read it, so we realized that it's added onto the entry, and the 15:43.880 --> 15:51.560 MPA, the entry, and then IP, and the pair. So this is the additional how the bytes are being stored 15:51.640 --> 15:57.720 into a 16-bit, a 16 bytes, I read, but only four bytes are being used to tow by discarded, 15:58.680 --> 16:03.960 and you think this is going to be a problem, but I think along the line, we copy when we are 16:03.960 --> 16:10.200 copying it into the buffer, we are going to copy the size of it, there's a four, the size of the, 16:11.480 --> 16:18.360 the MVP, the size of the IP, so there's 12 bits that are not actually copied. So, so now, 16:19.320 --> 16:22.920 let's look at the package with two, so we are packaging the good time to the buffer, 16:22.920 --> 16:27.720 what's up is that we first pack the root and the list header, which gives the whole idea of 16:27.720 --> 16:35.000 that the cluster of ND lists, which gives us information about it, and then when we are done, 16:35.000 --> 16:41.480 it gives us, we get the MVP header of it, so for others who have the whole ambulance, 16:41.560 --> 16:48.200 who do contain five MB pages, is the ID, our name, our version, and then our table, 16:48.200 --> 16:54.760 and then you'll continue the IP address of it, so for this five MB, there was actually a good one 16:54.760 --> 17:03.320 is that they answer into a whole loop, where the type is actually checked for each MVP, 17:03.320 --> 17:08.280 and then there's going to be a package of piece on that type, and then stored into the buffer. 17:09.080 --> 17:17.960 So now we're going to look at the original simple packing, and then we see why our tables are keeping 17:17.960 --> 17:23.560 so, so, so long, notice our tables were conditioned to keep so, so, so, so long to 17:23.560 --> 17:30.920 actually serialize in the buffer. So, we have always the size of the ND lists, and we allocate 17:31.640 --> 17:37.880 the database on the size into an unsigned buffer, and then we set an active pointer to the buffer, 17:37.880 --> 17:44.760 where we actually do how many places we set our data into the buffer. So, imagine we have the 17:44.760 --> 17:52.760 ambulance header, which is going to print the whole data, we have a 24 by 19 bytes, memory fields, 17:52.760 --> 17:59.560 and five-parted bytes, and so far as we copy, we perform the immune coefficient of this data 17:59.560 --> 18:04.440 to the staff not just of an active pointer, but to support the staff not just of the buffer. 18:05.400 --> 18:11.800 And so, we perform the immune coefficient of this into the active pointer, and then we add 18:11.800 --> 18:17.400 our trees, the pointer, in the steps of the size of the new buffer. So, it points to the new 18:17.400 --> 18:25.320 available memory buffer. And then, so, when we add and we set a left scantahoot to show us 18:25.320 --> 18:33.240 our data, let's not buffer, or we do not go. The less scantahoot decreases in steps of the bytes, 18:33.800 --> 18:40.920 so that when it gets zero, we reach a certain level, and we are not able to 18:40.920 --> 18:47.400 add any more bytes to the data buffer. So, after this, ambulance header has been copied into 18:47.400 --> 18:51.560 it this way, we have now another new pointer point, this is the next available free space. So, 18:51.560 --> 18:59.960 we are done, we increase our data, and then we decrease our left over. So, we actually, 18:59.960 --> 19:06.040 after that, we pack the MVP header. So, page MVP, we pack a header. The header being 19:06.040 --> 19:11.560 admitted data on information about the MVP, which gives me the number the MVP about to pack. 19:11.560 --> 19:20.760 And so, if we enter any header also, it's 24 bytes 19 bytes of 19 bytes of 19:21.480 --> 19:28.360 memory memory memory and five-parted bytes. So, we also pack the MVP header by 19:28.360 --> 19:37.160 admin copy operation. We copy the mb header into the buffer, that's going to be 19:37.160 --> 19:42.680 must be the canal. And then when we are done, we also perform an admin copy of the name, 19:42.680 --> 19:47.880 if you know the MVP, the name, if you know the name, we also being copied into the buffer. 19:47.880 --> 19:57.320 And then when we are done, we come and also copy the value, value part of the MVP into the 19:57.320 --> 20:03.480 buffer. All true copy operation, and then after that, we increase our size of our pointer, 20:03.480 --> 20:09.880 our page, we increase our size of our pointer, and then we decrease the size of our size of 20:10.200 --> 20:18.280 our pointer. So, how this is going on so this, then you what we have, after our phasing copy, 20:18.280 --> 20:26.840 so we copy this, after our first ambulance header, and then we added our MVP header for the 20:26.840 --> 20:34.280 first MVP, which is a fashion in 22, and then we added that we add the version to that for the 20:34.280 --> 20:39.240 header. We add the version, and then our first copy, our pointer will be here, and the 20:39.320 --> 20:45.400 pointer is supposed to a new memory in the buffer region. So, when data is being added, it's been 20:45.400 --> 20:49.720 pushing to the address of the start, the start is not the address of the points of the attitude 20:49.720 --> 20:57.800 points are. And so when we are done, realize that we now pack the value of the MVP into 20:57.800 --> 21:03.000 its true immunocopy operation, and then we add that we increase the pointer in size of the steps 21:03.000 --> 21:09.480 of the size of the value in bytes, and then we decrease our left pointer, and then so we realize 21:09.480 --> 21:17.000 that maybe the version was 22, so now we pack 22, and 22 is starting a new N64 type. So, 21:17.000 --> 21:23.800 on page, byte, memory will just turn the value 22, 22 is actually just used by identifying 21:23.800 --> 21:31.320 four bytes, you use 45 bits, 45 bits of a six-folded stream, and we actually copied the 21:31.400 --> 21:40.760 code of this version, but I also possible that also can't see the performance. And when we are done, 21:41.640 --> 21:49.160 I've copied the V, and so after we copied the value 22, that's what actually happens, just to copy 21:49.160 --> 21:58.040 a single MVP. Copy the whoever this structure, and then copy an MVP structure for the MVP, 21:58.040 --> 22:05.160 and then, after that you copy the value portion, the name fits, and the value all on that, 22:05.160 --> 22:15.960 and then copy your machine, and then, and then, and then, and then, we move the pointer, 22:15.960 --> 22:22.440 we actually go to the next MVP, and then, for the next example, like the name, and the blackness 22:22.520 --> 22:30.280 MVP, we actually copied the name of V. So, we actually copied the header of this name, 22:30.280 --> 22:35.960 blackness, and V. into the buffer, and then, and then, and then, we copied the name portion of 22:35.960 --> 22:41.000 it into the buffer, and then, we are done, we copied the value portion of it into the buffer, 22:41.000 --> 22:49.240 and all of this is followed by an increments in the pointer, in the calculator, so that we can 22:49.320 --> 22:54.520 point to a new starting point, a new freshman year, which is on use, so we can use to allocate 22:54.520 --> 23:02.360 to receive new items in the memory, so this was actually going on in the MVP, and to this is the 23:02.360 --> 23:11.320 whole portion of their insertion into the buffer, after we click about this, and so, this is the whole 23:11.400 --> 23:20.440 section of it, so, for just a single pack, we actually have about, I think, six, I mean, copies in the 23:20.440 --> 23:28.680 seven to eight, I just change, and we have, like, increments and increments, and, um, and then, so, 23:28.680 --> 23:34.440 for this one, like, this, the name, blackness, we copied the MVP, header, and then, we copied the name, 23:34.520 --> 23:39.880 version of it, and they will copy the, the value version of the MVP, then, where we are done, 23:39.880 --> 23:47.560 we increment the pointer to point to the new address, which is on use, buffer, in the buffer, 23:47.560 --> 23:53.400 which is so, it comes to store new data, new parameters, so, we realized that for our memory, we 23:53.400 --> 23:59.000 didn't become like this, and then, server operations, just for this, just for this, just for two 23:59.000 --> 24:08.360 MVP's, so, imagine if you have, like, 52 key IPs, we are actually packing, or far, I did a 24:08.360 --> 24:16.040 chase on it, and for 52 key, I visually are actually producing over 180,000 MVP's, and so, for each 24:16.040 --> 24:23.720 MVP, we pack an MVP header, and then we also pack the MVP name, there you also pack the MVP 24:23.720 --> 24:31.720 value, so, for each MVP header, we have three packets, and then, for each, and we have, for each 24:31.720 --> 24:41.560 MVP header, we have three packets, and each pack performs quite not expensive copies, but numerous copies, 24:41.560 --> 24:52.600 and so, this is consumed in a raw loop of the number of NDP's we have, so we have 180,000, we are going 24:52.600 --> 25:03.400 to a 180,000 loop of NDP packet, and that is what's actually going on, when we are sending 25:03.400 --> 25:04.920 serializing data to the kernel.