WEBVTT 00:00.000 --> 00:15.200 Well, good afternoon, and please welcome Strik Viln, Vilname, Vilname, I'm sorry. 00:15.200 --> 00:19.520 And he will talk about stats, baby stats roll. 00:19.520 --> 00:22.520 That's very rock, right? 00:22.520 --> 00:24.520 Please join me. 00:24.520 --> 00:40.120 Okay, hello, for us then, very happy to be here, and to make it a perfect solution for me today, 00:40.120 --> 00:44.480 my laptop managed to get some failures, so I have no keyboard. 00:44.480 --> 00:50.960 And so it was like a little bit stressful just before I have to replay things, and I hope 00:50.960 --> 01:01.280 the website and everything will be okay, so I'm sorry, Vilname, but apparently I'm missing 01:01.280 --> 01:12.200 the bottom of the slide already, so, well, it's supposed to be a bottom, I hope 01:12.200 --> 01:16.360 wasn't exactly the work, but so I'm sorry, I'm sorry, I'm not involved in the progress 01:16.360 --> 01:23.200 well long time, nearly 20 years, I started working on a full-text search online when I was in 01:23.200 --> 01:29.560 a web industry, and so I was involved in GSAW, and then I found it's a conference in France 01:29.560 --> 01:35.200 and in the 15 years ago, which has been a re-name, and we've earned it at that early 01:35.200 --> 01:42.640 four years ago, and so I'm not a CEO, President of Data Bene, and Data Bene is a service provider 01:42.640 --> 01:49.840 for the security and site to data, and we're providing many services, as many people, many 01:49.840 --> 01:53.640 restrictions, maybe for Data Bene, that we are not a software editor, we do only open source 01:53.640 --> 01:57.960 of hardware, so we are not a sensing, we are not selling license or anything, just providing 01:57.960 --> 02:03.760 services, and assistance, and trying to make you use the security and ecosystem the best 02:03.760 --> 02:10.720 way you can, and we are doing and running around here in software and also starting new 02:10.720 --> 02:20.120 project for our, and we are in, so the main point today we'll be to talk about the 02:20.120 --> 02:27.920 process here at 18 new things with statistics, and we will start by presenting what 02:27.920 --> 02:32.760 there was a cumulative statistic at the moment, is that already required, what's the importance 02:32.760 --> 02:41.360 for the security, and brief history, review of what happened on doing the equations, and 02:41.360 --> 02:46.000 some open question already at this point, and then how it works, and there's a wood in 02:46.000 --> 02:52.920 a C part, and so how we can use what's going to be possible in a possible 18, so at the 02:52.920 --> 02:57.320 end of the year, so you can write your own extension to manage your own statistics and 02:57.320 --> 03:07.000 metrics inside of the security, but using the potential n-rine to manage that for you. 03:07.000 --> 03:13.440 So cumulative statistics, you said, it was very, probably very correct at the beginning, 03:13.440 --> 03:20.000 but today, this is not absolutely all the view, but there is in the documentation of 03:20.000 --> 03:26.640 the SQL page, informing you of the functions and view, which are related to cumulative 03:26.640 --> 03:31.920 statistics, and so at the moment, there are many view, you probably know, if you're 03:31.920 --> 03:36.720 already used for the SQL, which are PG Stat activity, for instance, which is in this 03:36.720 --> 03:41.240 page, but it's maybe not really cumulative statistics still. 03:41.240 --> 03:45.880 We have replication, we have stuff for SSL, we have stuff for Baron Rite, for check 03:45.880 --> 03:52.760 point, right, for many things that cover it, and PG PG PG, and line is very rich, I mean, 03:52.760 --> 03:58.200 they often people ask how to get information in PG and very often just offered directly 03:58.200 --> 04:03.680 by the system, not everything, but there is a lot, a lot of views and functions to get 04:03.680 --> 04:07.080 a lot of information about what's happening on the Indian line. 04:07.080 --> 04:16.320 So, in order to get some example, I hope it's really good for you, but so the first 04:16.320 --> 04:20.160 view, first to act, it's just a view from the checkpointer, so Pitches that checkpointer 04:20.160 --> 04:26.400 in particular, which is a way to be informed about what is doing the checkpointer, or 04:26.400 --> 04:32.280 many time it has been started, triggered by timer, because you consume all space, what 04:32.320 --> 04:39.200 was the time writing, duration, and so on, okay? So clearly, this part is a community 04:39.200 --> 04:42.360 view, because you see it in a time, every time you have a new checkpoint, you make a 04:42.360 --> 04:47.160 plus one, and plus one, you increment, and you increase. If you consume time to write 04:47.160 --> 04:51.440 stuff, and you keep on causing more time, so the right time is going to increase of a 04:51.440 --> 04:59.680 time. This is expected. Just under, this is a much larger one, which is Pitches that 04:59.680 --> 05:06.880 you use your tables. This one is a little bit different, because this is a lot of cumulative 05:06.880 --> 05:11.800 statistics, but the use of tables are very important for the school and system tables 05:11.800 --> 05:16.520 overall, and so many of the information here are drawing with information that we 05:16.520 --> 05:20.120 grab when we are working on a table. I mean, when we're planning the query and we're 05:20.120 --> 05:25.560 doing things, we get from information, and from the same place, we can move from 05:26.520 --> 05:34.600 to the school and the table and the same table. So here, for example, you see that 05:34.600 --> 05:42.080 is a interesting line here. We tell you that there is one million rows in the table, 05:42.080 --> 05:47.760 but then if you go there, there is no delete note, there is nothing, but still the 05:47.760 --> 05:54.480 live purpose are not one million, because this one information is not true. This is just 05:54.480 --> 06:00.080 an estimation of what's inside, okay, we'll get back to this point. So cumulative 06:00.080 --> 06:05.040 statistics and set system, it's not always exactly what you may think it is really 06:05.040 --> 06:13.720 aware. And in order to get some more example, the Pitches that activities that you probably 06:13.720 --> 06:17.560 know, this is the top-like for the school, so we're just joining with a PID of your 06:17.560 --> 06:25.000 process and informing you of bootware and new back-and-started what they're doing and so on. 06:25.000 --> 06:33.280 And you even have new view like this one where you're able to track what's going on. 06:33.280 --> 06:38.080 You're earning a create index or you're earning a vacuum or some operation. You have a 06:38.080 --> 06:43.120 views which are going to inform you where are you in this process, at what stage of 06:43.120 --> 06:48.360 the creation of the index you are and what's going on. So this is cumulative, but just 06:48.360 --> 06:53.400 for me, just to look at some activity at a point in time and then it's flush and the view 06:53.400 --> 07:06.760 will be empty on the index creation has been completed. So, on those you, many are way 07:06.760 --> 07:11.000 how cumulative statistics, so community statistics, it's really just a counter and you 07:11.000 --> 07:19.920 increment. Okay, but there is a special one, I mentioned, Pitches that all tables, which is 07:19.920 --> 07:26.080 also Pitches that all tables, system tables and user tables. This is the same thing, all 07:26.080 --> 07:30.800 is everything and then you have view to just look at a system of just look at user 07:30.800 --> 07:36.800 tables. So, except it's one which is very special, all the 07:36.800 --> 07:44.440 view mentioned in just cumulative statistics as usual. Okay, and then we have all the views which 07:44.440 --> 07:50.680 are more about activity and joining statistics, cumulative statistics and also current activity. 07:50.680 --> 07:58.920 What's going on? And then we have a lot, much more new, which are only about current activity. 07:58.920 --> 08:03.480 So, this is cumulative statistics in the documentation, but in fact, it's not always 08:03.480 --> 08:10.120 cumulative statistics, just monitoring information, ongoing information and we keep all 08:10.120 --> 08:16.040 the gaps in the name that was introduced in Portugal maybe 30 years ago and which maybe 08:16.040 --> 08:24.040 is not a great anymore. I want to talk to much about that, but there are also many 08:24.040 --> 08:30.320 functions, because all the views, others use and so what I do is that for each field, 08:30.320 --> 08:34.560 each attribute, which goes going to call function, which is in the C function, which is 08:34.560 --> 08:40.320 going to provide the value to put in the view. So, views are just a collection of functions 08:40.320 --> 08:45.440 and other would. There are more functions than the views, because we have functions with 08:45.440 --> 08:50.480 statistics to clean the cache of stats and some few functions which are going to be added 08:50.480 --> 08:59.480 during political 18. And so, the first thing I believe which is super important to understand 08:59.480 --> 09:05.480 is that we have also data statistics in political, this is super important, the data statistics 09:05.480 --> 09:11.480 is what's inside your tables, inside your columns. So, this is what is used by political 09:11.480 --> 09:16.280 to evaluate the distribution of the data and to ensure that you make a good choice when 09:16.280 --> 09:22.280 you do an in-exclan or sequential scan and what's where. But those are user statistics, 09:22.280 --> 09:28.280 user data and it's very confusing because the name is the same. So, it's PG stats and 09:28.280 --> 09:39.040 PG statistics, but it's not what we are talking about today. And for memory, I put two 09:39.040 --> 09:45.040 part for users and for systems, because by default, political have the machine access to 09:45.040 --> 09:51.720 those statistics. But when a user want to look at the statistics that has been produced 09:51.720 --> 09:59.040 by analyze, we use the view for users. So, we respect the fact that this user is granted 09:59.040 --> 10:10.040 to access this data. It's respecting the grant and we work of each user as they can access. 10:10.040 --> 10:15.360 So, first thing that I'm really tempted to push on the word matrix which is maybe more 10:15.360 --> 10:22.040 accurate today would what we use to do on my drink system both use. And so, matrix are not 10:22.040 --> 10:28.040 to weight. It's not absolutely true. There should be free weight, but they are not as confidential 10:28.040 --> 10:36.040 as free weight as statistics are. And so, there is a big distinction here. And so, the 10:36.040 --> 10:44.040 communicative statistics allows you to get inside, inside of would say and line, I'm 10:44.040 --> 10:51.040 getting dashboard, getting fancy matrix, a boot squatting on, no buff in saturion and so on. 10:51.040 --> 11:01.040 Okay? One, maybe key aspect for today that there is many service providers for writing 11:01.040 --> 11:07.040 position on the cloud. And I'm very, I mean, you know, as you and so on, but also on private 11:07.040 --> 11:12.040 cloud, I mean, for example, for the scenario with also managing its own cloud, there 11:12.040 --> 11:18.040 also need to provide information for users of the online. But there is the extension 11:18.040 --> 11:23.040 between being able to provide monitoring information and getting access to that database 11:23.040 --> 11:27.040 and access data. And so, the monitoring and the communicative statistics is very, very 11:27.040 --> 11:33.040 a way to segregate between what we can access and what we cannot access when we do monitoring. 11:33.040 --> 11:40.040 That's a very big important point. And so, the matrix for those service providers, especially 11:40.040 --> 11:47.040 in the cloud area, are very important. Probably that it has justified, so many activities 11:47.040 --> 11:56.040 of four last years in this area. And matrix are essentially, if you want to get a good view 11:56.040 --> 12:01.040 on what going on in this system, if you want to discover problems, you don't have a button 12:01.040 --> 12:09.040 neck and so on or debug, matrix like those are very, very interesting. So it's just a summary. 12:09.040 --> 12:25.040 And so, as I said today, we're going to talk about this part. So, communicative statistics 12:25.040 --> 12:30.040 are like matrix, and there are not too close to impact how close the school is working. 12:30.040 --> 12:36.040 It's behavior, but it's not always true. This is true for 99% of the data we have, but only for 12:36.040 --> 12:43.040 that. And as I said, communicative statistics can be information about ongoing activity. 12:43.040 --> 12:47.040 And there are not really directly impacting the school behavior, but still, it will be interested 12:47.040 --> 12:53.040 by that because, for example, for replication purpose, when you want to do a switch over 12:53.040 --> 12:58.040 or fail over, you may want to fail over to the replica, which is the most advanced to the 12:58.040 --> 13:06.040 replication stream. And so, to be aware that you look at those stats. The many big things 13:06.040 --> 13:12.040 is very far out of our room, the boring point I will say. Out of our room is a process 13:12.040 --> 13:16.040 we're going to do maintenance on your database, on your tables, running and lies, 13:16.040 --> 13:20.040 and on your vacuum. And this is triggered by a number of rules, which has been 13:20.040 --> 13:26.040 modified, inserted, deleted, and so on. And those information are coming from the 13:26.040 --> 13:31.040 critical, cumulative statistics system. So there is nothing else because if you visit 13:31.040 --> 13:47.040 the stats, it will come on fork. This is created right in the documentation of the 13:47.040 --> 13:52.040 school. This is a warning from the doc of the school. Using PC stat Reset, also with 13:52.040 --> 13:55.040 a computer that the computer can use to determine when to trigger a document on an 13:55.040 --> 13:59.040 a lies. With the things the screen tells can cause otherwise come to not pay for 13:59.040 --> 14:04.040 this sidewalk, which can cause problems such as table blood or outdated table statistics. 14:04.040 --> 14:08.040 How? The tab as the word, analyze, recommended after statistics, I've been 14:08.040 --> 14:14.040 visited. Okay. So it means that when you restore your database, usually 14:14.040 --> 14:20.040 after PC restore, you run and analyze because of that. If you reset your stats, you 14:20.040 --> 14:24.040 still need to run and analyze. Running and analyze on one gigabytes of data or 14:24.040 --> 14:29.040 one terabyte of data or ten gigabytes of data is not the same story. And so maybe 14:29.040 --> 14:33.040 what was acceptable before is not acceptable anymore. Because sometimes we need to 14:33.040 --> 14:37.040 reset start to get a better view on system because the workload has changed 14:37.040 --> 14:41.040 over time. And so you have two big numbers. It's not interesting. It's not moving 14:42.040 --> 14:46.040 well for you. So you want to reset the start. But if your database is very large, 14:46.040 --> 14:51.040 you're going to impact your database a lot. So getting split here will be probably 14:51.040 --> 15:00.040 very interesting. I've got a brief story about what I've planned on doing 15:00.040 --> 15:06.040 the so-and-use in the school in the server. So the big change has been in 15:06.040 --> 15:12.040 in the school 15. Because before the school 15, the statistics were on a file. 15:12.040 --> 15:17.040 So it means that each 50 minutes a month or I don't remember exactly it was just 15:17.040 --> 15:23.040 rushing to file. So you add this down stats term directory and you pack 15:23.040 --> 15:28.040 a and then you copy and you pack a and you copy. And for Windows user, it was 15:28.040 --> 15:32.040 one of the first boring things because Windows user add to move the stats 15:32.040 --> 15:36.040 time directory in a RAM drive in order to get something which was 15:36.040 --> 15:44.040 efficient. And it was not very robust. And also it required to delay 15:44.040 --> 15:48.040 the amount when you can access the information. Because you cannot get the 15:48.040 --> 15:53.040 new stats until it has been flushed to disk. So it wasn't very big 15:53.040 --> 15:57.040 things to move that from the file system to the memory only. And so it has been 15:57.040 --> 16:07.040 run and possibly 15. And so we fortunately we dropped this one, which is 16:07.040 --> 16:14.040 actually nice. But we added this one that nobody knows. And this one 16:14.040 --> 16:19.040 parameter here, stats based on C, is the parameter which is going to 16:19.040 --> 16:25.040 inform when you connect. If a new session you're going to require statistics, 16:25.040 --> 16:29.040 what are you going to do with the stats? Do you want to get fresh 16:29.040 --> 16:34.040 stats every time? So every time you get the number of 16:34.040 --> 16:38.040 insert in the table, you get back to the main memory and get the 16:38.040 --> 16:43.040 information for each get. Or at the first time that you get this 16:43.040 --> 16:46.040 information for this table, you keep it in cache in your local 16:46.040 --> 16:54.040 section. Or as soon as you start using stats, you grab all 16:54.040 --> 16:59.040 the cache from the main memory to your back end. And the three are 16:59.040 --> 17:03.040 interesting. The first one where you fetch every time is very 17:03.040 --> 17:06.040 interesting for monitoring systems. Because it means that you can 17:06.040 --> 17:10.040 be connected on the processure with your monitor and just get 17:10.040 --> 17:14.040 the stats and you know it's fresh that every time because it's coming from the main memory. 17:14.040 --> 17:18.040 No problem. The second one is very interesting and this is a default 17:18.040 --> 17:22.040 it's called cache. Because when we plan the query or when we do 17:22.040 --> 17:24.040 processing, in particular we want 17:24.040 --> 17:28.040 stable statistics at one point. When you're planning a 17:28.040 --> 17:32.040 join between tables and simple, it will be the same table at several times. 17:32.040 --> 17:36.040 We don't want the stats to change as we're planning the query. 17:36.040 --> 17:40.040 And the last one. But I'm not sure exactly what the very 17:40.040 --> 17:44.040 piece of it. But it exists. So you get all the cache, all the stats 17:44.040 --> 17:48.040 locally. Be aware that the number of stats are very 17:48.040 --> 17:52.040 changed over time. And years ago we had maybe 10 or 20 17:52.040 --> 17:56.040 kilobytes of stats. So getting a full copy was very 17:56.040 --> 18:00.040 not a problem. Today, with what we have in political 18:00.040 --> 18:05.040 17. And next, a risk political 18, we have statistics 18:05.040 --> 18:08.040 per table, per back end, and a lot of for IO, far 18:08.040 --> 18:11.040 racing. And currently, if you're one million 18:11.040 --> 18:16.040 tables, you're one gigabyte of stats associated with 18:16.040 --> 18:20.040 this one million tables. So it means that if you have 18:20.040 --> 18:24.040 selected fetch consistency to grab everything, every time you 18:24.040 --> 18:28.040 connect something, you have one gigabyte of memory back in your 18:28.040 --> 18:30.040 back end. So you connect 10 back end, and you can 18:30.040 --> 18:33.040 send 10 gigabyte. There is no copy on the right 18:33.040 --> 18:37.040 of whatever. So be careful of this one. 18:37.040 --> 18:42.040 So we're improving on after that much more. I mean, 18:42.040 --> 18:45.040 starting with the fact that stats are in my way, we are very 18:45.040 --> 18:48.040 free to do more things. It's much more efficient. And some 18:48.040 --> 18:51.040 kind of add more stats, and using more effectively. So 18:51.040 --> 18:57.040 there were a lot of interest, facility for IO, which was very 18:57.040 --> 19:00.040 super important to get more information. IO in 19:00.040 --> 19:03.040 particular, has always been a pain. Be aware, I am 19:03.040 --> 19:06.040 waiting for memory, or I am waiting for the system, or 19:06.040 --> 19:10.040 from this, or I would not want to take to get this 19:10.040 --> 19:14.040 this buffer field and so on. So getting views on IO 19:14.040 --> 19:17.040 is very interesting, and so you should step forward, and 19:17.040 --> 19:22.040 also consuming memory. And there are also attributes. 19:22.040 --> 19:26.040 I need to pitch that tables, pitch that indexes, to get 19:26.040 --> 19:29.040 a lot of information about the last index, or the last 19:29.040 --> 19:33.040 access to logic. It's interesting if my index has not been 19:33.040 --> 19:37.040 accessed since maybe one year. Maybe it's safe to 19:37.040 --> 19:41.040 get that this index is not really required to accept 19:41.040 --> 19:48.040 it for a constraint. So it helps the operation and 19:48.040 --> 19:52.040 DBA to discover, use an unused object in your 19:52.040 --> 20:01.040 online. In postcards, I don't list everything because 20:01.040 --> 20:07.040 well, it's not supposed to be. Postcards 17, we had 20:07.040 --> 20:11.040 it, which was already presenting in a big 20:11.040 --> 20:14.040 variety of view, but we split both. So it's clear. 20:14.040 --> 20:16.040 This is a background right of postcards, or this is a 20:16.040 --> 20:18.040 background, track point of postcards, and get 20:18.040 --> 20:21.040 distant statistics, distant information. So for 20:21.040 --> 20:25.040 operation, it's very efficient, very much more easier than 20:25.040 --> 20:29.040 before, and much more interesting also. And 20:29.040 --> 20:33.040 events that increase number of statistics, we also 20:33.040 --> 20:36.040 increase the way we want to use that statistics. At the 20:36.040 --> 20:39.040 beginning, we had only one big function, just 20:39.040 --> 20:42.040 reset everything. And after that, we start saying 20:42.040 --> 20:44.040 no, no, we cannot reset everything every time, we 20:44.040 --> 20:48.040 just want to reset some chat catalogs or just some 20:48.040 --> 20:51.040 local statistics. Time for exam, and in postcards 20:51.040 --> 20:54.040 17, it has been proved to be able to reset very 20:54.040 --> 20:57.040 just small part of your stats. So you do not have to 20:57.040 --> 21:04.040 analyze every time. And in postcards 17, I believe 21:05.040 --> 21:07.040 this one was very interesting, which is started 21:07.040 --> 21:10.040 about reporting with the progress on a 21:10.040 --> 21:12.040 person. Back home was a little bit before I 21:12.040 --> 21:15.040 think, but as index processing and use maintenance, 21:15.040 --> 21:18.040 because always a question you're creating an index, and 21:18.040 --> 21:21.040 you don't know when you're going to end. With those 21:21.040 --> 21:24.040 view, you have a view of where it is, and you 21:24.040 --> 21:27.040 can evaluate how much work it remains to do. 21:27.040 --> 21:36.040 So, here, I want you to put what has been 21:36.040 --> 21:40.040 I did so far in postcards 18, but it's moving a lot. 21:40.040 --> 21:43.040 When I say it's moving a lot, I prepare those slides at 21:43.040 --> 21:46.040 the beginning of January. And basically, I have to read 21:46.040 --> 21:50.040 right part of them because it has been time. And I 21:50.040 --> 21:54.040 is ongoing commit. More or less, I believe the more 21:54.040 --> 22:00.040 important thing is that in postcards 18, so here is my 22:00.040 --> 22:05.040 slide. In postcards 18, we are going to have a tracking of 22:05.040 --> 22:09.040 stats for pairback and for many things. So instead of getting 22:09.040 --> 22:12.040 for example information about the world stats system, a 22:12.040 --> 22:15.040 wall did you consume overall. You will be able to know 22:15.040 --> 22:18.040 how much wall you consume on this back end. You 22:18.040 --> 22:21.040 can use this action, this operation. And this will be 22:21.040 --> 22:24.040 valid for a lot of things. So starting with next 22:24.040 --> 22:28.040 version, we will have more tools to be able to 22:28.040 --> 22:34.040 profile the your session and your activity. 22:34.040 --> 22:40.040 It also first, what is currently committed. It also 22:40.040 --> 22:45.040 first to get session tracking, we already have 22:45.040 --> 22:49.040 transaction tracking so far. But here's a new 22:49.040 --> 22:52.040 commit, which are included. Put everything in the 22:52.040 --> 22:55.040 system to make it much more easier and future to 22:55.040 --> 22:58.040 add new stats and to be able to fine tune what you 22:58.040 --> 23:05.040 want. Okay. This slide was just to outline the 23:05.040 --> 23:10.040 three main problem we have, which is that 23:10.040 --> 23:13.040 details, number of insights in the back 23:13.040 --> 23:15.040 room and number of modifications in the 23:16.040 --> 23:19.040 system, which are used by the 23:19.040 --> 23:23.040 person. Other than that, not a lot of single 23:23.040 --> 23:27.040 critical. Also, it's called 23:27.040 --> 23:30.040 administrative statistics, but in fact, 23:30.040 --> 23:33.040 sometimes it's just a go. So it's not 23:33.040 --> 23:35.040 communicative, sometimes it's just not a 23:35.040 --> 23:38.040 go to a go. So again, we just use what was in 23:38.040 --> 23:41.040 pace, keeps the same wording, same name, but 23:41.040 --> 23:44.040 the use that has changed over time. So I 23:44.040 --> 23:46.040 believe for next version, if we're 23:46.040 --> 23:48.040 interesting to align here and get 23:48.040 --> 23:50.040 information in the documentation to 23:50.040 --> 23:53.040 better outline what is statistics, what is 23:53.040 --> 23:56.040 committed, what is the goge, what is 23:56.040 --> 24:06.040 activity. So my open question for 24:06.040 --> 24:08.040 developers of the school and 24:08.040 --> 24:11.040 committers. Because in 24:11.040 --> 24:14.040 the school 18, you will be able to write 24:14.040 --> 24:16.040 your own extension to manage your own 24:16.040 --> 24:18.040 statistics. So the statistics, which is 24:18.040 --> 24:20.040 in memory and which is internal to 24:20.040 --> 24:22.040 the school at the moment, we will be 24:22.040 --> 24:24.040 also with an API in the future. 24:24.040 --> 24:26.040 Starting with that, it means that people 24:26.040 --> 24:28.040 are going to do the development, and so 24:28.040 --> 24:31.040 it will be very hard to change 24:31.040 --> 24:34.040 function name, view names, or even 24:34.040 --> 24:37.040 a lot of things. So I will be 24:37.040 --> 24:39.040 before the school 18 is released. 24:39.040 --> 24:41.040 We need to ensure that what we have 24:41.040 --> 24:43.040 makes sense in the future, and that we do not 24:43.040 --> 24:45.040 make the same design mistakes that we made 24:45.040 --> 24:48.040 some years ago when we were thinking 24:48.040 --> 24:50.040 because of the help of I, or that 24:50.040 --> 24:52.040 the number of blocks changing was 24:52.040 --> 24:54.040 enough, and we realize that no, it's 24:54.040 --> 24:55.040 not enough, we need to know the 24:55.040 --> 24:57.040 bytes, which has been modified, which is 24:57.040 --> 25:01.040 this time. So we are adapting. 25:02.040 --> 25:05.040 So my point is to try to remove 25:05.040 --> 25:08.040 this piece of that view at all, because 25:08.040 --> 25:10.040 piece of stats and piece statistics are 25:10.040 --> 25:11.040 user data, and we have something with 25:11.040 --> 25:13.040 similar name, which is something distinct. 25:13.040 --> 25:16.040 So let's remain in, maybe we need 25:16.040 --> 25:18.040 piece of metrics, or piece of metrics, or 25:18.040 --> 25:20.040 whatever name, which is very clear, 25:20.040 --> 25:23.040 this is not a user stats. 25:23.040 --> 25:26.040 And also the point of 25:26.040 --> 25:29.040 boots, auto vacuum, and the number of 25:29.040 --> 25:31.040 tupper in the table. 25:31.040 --> 25:33.040 Years ago, maybe a decade ago, 25:33.040 --> 25:35.040 it has been a really long discussion about 25:35.040 --> 25:37.040 how to accurately report 25:37.040 --> 25:39.040 the number of four in the table, and how to 25:39.040 --> 25:41.040 accurately report the number of 25:41.040 --> 25:43.040 insert and so on. 25:43.040 --> 25:46.040 And it had lead it to new 25:46.040 --> 25:48.040 algorithm into possession field, and 25:48.040 --> 25:49.040 when you run analyze and vacuum, 25:49.040 --> 25:51.040 it's going to re-epaluate the number of 25:51.040 --> 25:52.040 fours in your tables every time, and 25:52.040 --> 25:53.040 reduce that. 25:53.040 --> 25:55.040 It was very important. 25:55.040 --> 25:56.040 I'm not sure it's so much 25:56.040 --> 25:58.040 important today, because before the 25:58.040 --> 25:59.040 numbers were not stable, 25:59.040 --> 26:00.040 they were not quite safe, they 26:00.040 --> 26:01.040 were not very good. 26:01.040 --> 26:03.040 Today, we are able to have 26:03.040 --> 26:05.040 stats with the notion of 26:05.040 --> 26:07.040 create, transactional, and 26:07.040 --> 26:08.040 what transactional. 26:08.040 --> 26:09.040 So it means that we can prepare 26:09.040 --> 26:11.040 stats, but if there is a rollback, 26:11.040 --> 26:13.040 we also rollback the stats, and 26:13.040 --> 26:14.040 so that that we may 26:14.040 --> 26:15.040 inaccurate. 26:15.040 --> 26:17.040 And so maybe there is a point 26:17.040 --> 26:18.040 here to try to re-work 26:18.040 --> 26:20.040 his part, and to use more 26:20.040 --> 26:22.040 transactional information, to 26:22.040 --> 26:23.040 figure what is the number of 26:23.040 --> 26:25.040 tuppers, which has been 26:25.040 --> 26:32.040 to then start auto vacuum. 26:32.040 --> 26:36.040 So how does that work? 26:36.040 --> 26:40.040 Starting in the C code more here. 26:40.040 --> 26:42.040 We use shared memory to keep 26:42.040 --> 26:43.040 shared stats. 26:43.040 --> 26:44.040 So shared memory is easy. 26:44.040 --> 26:46.040 This is a nice place in 26:46.040 --> 26:48.040 RAM, that many process can access, 26:48.040 --> 26:50.040 and get information from it, 26:50.040 --> 26:53.040 or modify it, if there is a lock. 26:53.040 --> 26:55.040 And we have locked memory to 26:55.040 --> 26:56.040 a community statistics. 26:56.040 --> 26:57.040 So in particular, you know, it is 26:57.040 --> 26:58.040 only forks. 26:58.040 --> 27:00.040 So when you have a new connection, 27:00.040 --> 27:01.040 you create a new forks. 27:01.040 --> 27:03.040 This is your backhand, and this backhand 27:03.040 --> 27:05.040 is going to be able to a community 27:05.040 --> 27:06.040 statistics and information. 27:06.040 --> 27:08.040 And at some point, given some 27:08.040 --> 27:09.040 condition, maybe there is 27:09.040 --> 27:10.040 another transaction, maybe a 27:10.040 --> 27:12.040 timeout, maybe a direct 27:12.040 --> 27:14.040 function call or whatever. 27:14.040 --> 27:16.040 These local stats are going to be 27:16.040 --> 27:18.040 merged or flushed into the shared memory, 27:18.040 --> 27:20.040 and so other people will access 27:20.040 --> 27:23.040 it also. 27:23.040 --> 27:25.040 And the other point is that 27:25.040 --> 27:27.040 we have statistics 27:27.040 --> 27:28.040 persistence on disk. 27:28.040 --> 27:30.040 As a moment, statistics are 27:30.040 --> 27:32.040 persisted when we shut down 27:32.040 --> 27:33.040 the school. 27:33.040 --> 27:35.040 If I'm not wrong, 27:35.040 --> 27:36.040 maybe there are ongoing work 27:36.040 --> 27:38.040 also to flush part of the 27:38.040 --> 27:40.040 start doing checkpoint, but I 27:40.040 --> 27:41.040 didn't follow up here, 27:41.040 --> 27:44.040 and I'm not absolutely sure. 27:44.040 --> 27:47.040 But that, the shared memory 27:47.040 --> 27:49.040 local memory is a flush 27:49.040 --> 27:54.040 and the start persistence makes 27:54.040 --> 27:56.040 all the system very robust and very 27:56.040 --> 27:59.040 also robust for concurrency. 27:59.040 --> 28:01.040 Because this is local to the backhand, 28:01.040 --> 28:03.040 you can manage the stats very 28:03.040 --> 28:05.040 freely without impacting other users. 28:05.040 --> 28:07.040 The only moment where there is concurrency 28:07.040 --> 28:09.040 and is when you merge your local 28:09.040 --> 28:12.040 stats with the main memory. 28:12.040 --> 28:14.040 So if we go into the C-part for 28:14.040 --> 28:16.040 extension developer, this is probably 28:16.040 --> 28:18.040 why you want to go to be able 28:18.040 --> 28:20.040 to write your own extension to manage 28:20.040 --> 28:22.040 your own statistics for your application, 28:22.040 --> 28:24.040 your device or system or whatsoever. 28:24.040 --> 28:26.040 At the end of the stats system from 28:26.040 --> 28:27.040 for the school, 28:27.040 --> 28:29.040 it's not as easy to complain the same way 28:29.040 --> 28:31.040 as the tables and the time tables are, 28:31.040 --> 28:34.040 but it's probably much more than enough 28:34.040 --> 28:36.040 to many metrics that you use 28:36.040 --> 28:38.040 to have for your application. 28:38.040 --> 28:39.040 So maybe in the future, 28:39.040 --> 28:41.040 instead of getting counters in your tables, 28:41.040 --> 28:43.040 you will have counters in the stats system, 28:43.040 --> 28:47.040 which is much, much, much more efficient. 28:47.040 --> 28:50.040 So basically, 28:50.040 --> 28:52.040 that you have C-parts, 28:52.040 --> 28:54.040 pgstat.c, 28:54.040 --> 28:55.040 pgstat functions, 28:55.040 --> 28:56.040 which are going to provide 28:56.040 --> 28:59.040 all the basic infrastructure. 28:59.040 --> 29:00.040 And you have then pgstat, 29:00.040 --> 29:02.040 underscore something for all 29:02.040 --> 29:04.040 the kind of statistics that you're going 29:04.040 --> 29:05.040 to manage. 29:05.040 --> 29:07.040 Table, function, database, 29:07.040 --> 29:08.040 background writer, 29:08.040 --> 29:10.040 check font writer and so on. 29:10.040 --> 29:14.040 And then we have iris here, 29:14.040 --> 29:17.040 which are the most important. 29:17.040 --> 29:21.040 The first thing is that pgstat internal 29:21.040 --> 29:24.040 is also for external process. 29:24.040 --> 29:26.040 So it's not really internal. 29:26.040 --> 29:27.040 This was internal, 29:27.040 --> 29:29.040 but this is either that you want to include 29:29.040 --> 29:30.040 when you write an end condition 29:30.040 --> 29:31.040 and you want to manage that. 29:31.040 --> 29:32.040 Okay? 29:32.040 --> 29:34.040 Because if you include pgstat internal, 29:34.040 --> 29:37.040 you grab everything you need to write your extension. 29:37.040 --> 29:39.040 So in those three, 29:39.040 --> 29:41.040 the rest you have definition about 29:41.040 --> 29:44.040 all the API structure that you have by default 29:44.040 --> 29:46.040 and the function that you have by default. 29:46.040 --> 29:48.040 Okay? 29:48.040 --> 29:51.040 And the new file appear very recently, 29:51.040 --> 29:53.040 which is pgstat kind, 29:53.040 --> 29:55.040 which is going to describe what kind 29:55.040 --> 29:57.040 of statistics you have in the system. 29:57.040 --> 29:59.040 Okay? 29:59.040 --> 30:01.040 And, 30:01.040 --> 30:04.040 dumb. 30:04.040 --> 30:06.040 Yes, this one. 30:06.040 --> 30:08.040 What? Sorry. 30:09.040 --> 30:11.040 Sorry. 30:11.040 --> 30:12.040 Starts kind. 30:12.040 --> 30:14.040 I go on a little off the other side. 30:14.040 --> 30:17.040 So the structure that you want to use 30:17.040 --> 30:19.040 when you write your extension, 30:19.040 --> 30:21.040 to manage that are very easy to understand. 30:21.040 --> 30:23.040 After all, it's very simple system. 30:23.040 --> 30:25.040 There is one first structure that you're going 30:25.040 --> 30:27.040 to use to store your metrics. 30:27.040 --> 30:30.040 You have free to do whatever you want. 30:30.040 --> 30:32.040 It's better for a bit of stick to the pgstat 30:32.040 --> 30:35.040 counter type, which is provided by a particular, 30:35.040 --> 30:37.040 but you can use other data type. 30:37.040 --> 30:41.040 Like Boolean, Joacking, Tegger, or whatever. 30:41.040 --> 30:43.040 Okay? 30:43.040 --> 30:45.040 So in this pgstat, 30:45.040 --> 30:47.040 start from country that we have an example. 30:47.040 --> 30:49.040 This is the one used by for the scale for managing 30:49.040 --> 30:51.040 the function's call. 30:51.040 --> 30:53.040 How many times this function has been called? 30:53.040 --> 30:55.040 How much time spent executing this function? 30:55.040 --> 30:56.040 Set time. 30:56.040 --> 31:00.040 I'm global time, including the ship course. 31:00.040 --> 31:04.040 So this structure will be used to store statistics. 31:04.040 --> 31:07.040 And this one will be used by possess 31:07.040 --> 31:10.040 to be able to get where your statistics are 31:10.040 --> 31:12.040 and to access it. 31:12.040 --> 31:13.040 Okay? 31:13.040 --> 31:17.040 So the first more less opaque while the second is very important 31:17.040 --> 31:19.040 and we don't need to manage this one. 31:19.040 --> 31:22.040 Sometimes it's interesting to add new members to this structure, 31:22.040 --> 31:26.040 but the default one is very valid for most statistics. 31:26.040 --> 31:31.040 Okay? And this is entry point for your local statistics. 31:32.040 --> 31:35.040 And then we have a pgstat entry ref, 31:35.040 --> 31:39.040 which is a generic structure to manipulate your stats. 31:39.040 --> 31:43.040 And so for possessual what he knows that there is no entry ref. 31:43.040 --> 31:45.040 In the entry of the pending entries, 31:45.040 --> 31:48.040 there is a current entry, there is a number of pending list actions 31:48.040 --> 31:50.040 in those that and they will use that. 31:56.040 --> 31:58.040 So the concept of kind, 31:58.040 --> 32:02.040 this is a list that you can find in pgstat.sh. 32:02.040 --> 32:06.040 And so here the first part here, 32:06.040 --> 32:09.040 which are viable number of objects. 32:09.040 --> 32:11.040 So this is a kind database, 32:11.040 --> 32:12.040 relation, functions, 32:12.040 --> 32:15.040 replications, replication slot, 32:15.040 --> 32:17.040 subscription and backends. 32:17.040 --> 32:19.040 All of that you can have several. 32:19.040 --> 32:21.040 It's not just one row. 32:21.040 --> 32:23.040 You will have one row, 32:23.040 --> 32:26.040 a one entry for each of your databases, 32:26.040 --> 32:28.040 relation and so on. 32:28.040 --> 32:30.040 So it's a viable number of objects that you have to manage 32:30.040 --> 32:32.040 to the stats system. 32:32.040 --> 32:35.040 In position, on the channel memory, 32:35.040 --> 32:38.040 there is a good algorithm which is going to scale up the memory 32:38.040 --> 32:41.040 and to increase the size of the decay to stats 32:41.040 --> 32:44.040 as when you increase the number of relation, 32:44.040 --> 32:46.040 or when you increase the number of databases, 32:46.040 --> 32:47.040 and you need more room. 32:47.040 --> 32:49.040 This is managed by possessure itself. 32:49.040 --> 32:52.040 You don't have to do anything. 32:52.040 --> 32:54.040 And there is another part, 32:54.040 --> 32:56.040 which is a good fixed number. 32:56.040 --> 32:59.040 So here this is often just for a single process. 32:59.040 --> 33:00.040 I'll try to go back on writer, 33:00.040 --> 33:02.040 check pointer and so on. 33:02.040 --> 33:03.040 So you know, 33:03.040 --> 33:06.040 there is as many numbers of row and no more. 33:06.040 --> 33:07.040 This is fixed attribute, 33:07.040 --> 33:09.040 a little bit more efficient maybe, 33:09.040 --> 33:11.040 but after all for us as a user, 33:11.040 --> 33:12.040 it's very similar. 33:12.040 --> 33:15.040 What you will change for accession developers, 33:15.040 --> 33:18.040 that's a function callback that we will see after that, 33:18.040 --> 33:19.040 another same, 33:19.040 --> 33:23.040 if you manage variables statistics of fixed number of statistics. 33:24.040 --> 33:27.040 And so the thing for possessure ageing is here, 33:27.040 --> 33:30.040 is that in possessure ageing, 33:30.040 --> 33:32.040 we can create new stats. 33:32.040 --> 33:34.040 Registrate to the system, 33:34.040 --> 33:37.040 and possessure is going to manage that for us. 33:40.040 --> 33:42.040 So in summary, 33:42.040 --> 33:44.040 we have global chart memory, 33:44.040 --> 33:48.040 we have local backend only memory, 33:48.040 --> 33:51.040 and we have a callback when we create new stats, 33:51.040 --> 33:54.040 and we say auto merge data for my local memory, 33:54.040 --> 33:55.040 to my main memory. 33:55.040 --> 33:57.040 How to read that the information is back, 33:57.040 --> 34:00.040 and what some few callback like this one. 34:02.040 --> 34:05.040 We also have some look, 34:05.040 --> 34:07.040 to manage cashing and snapshotting of the stats, 34:07.040 --> 34:09.040 as I saw before, 34:09.040 --> 34:11.040 the stats page consumption, for example. 34:13.040 --> 34:17.040 And this notion of local backend only memory, 34:17.040 --> 34:18.040 and global memory, 34:18.040 --> 34:20.040 if you're looking at that for my money, 34:20.040 --> 34:23.040 to write a point of view from a telemetry point of view, 34:23.040 --> 34:25.040 immediately you will discover that 34:25.040 --> 34:29.040 that there is notion about synchronous or asynchronous metric. 34:29.040 --> 34:31.040 And if you are dating directly, 34:31.040 --> 34:33.040 the share memory is a global one, 34:33.040 --> 34:35.040 this is synchronous metric probably, 34:35.040 --> 34:38.040 or at least you are able to provide some single synchronous. 34:38.040 --> 34:40.040 But if you go through the local backend only, 34:40.040 --> 34:42.040 this asynchronous metric. 34:42.040 --> 34:44.040 So if you use an open telemetry, 34:44.040 --> 34:45.040 for example, 34:45.040 --> 34:47.040 this distinction is very pertinent, 34:47.040 --> 34:49.040 and it's reset here. 34:49.040 --> 34:51.040 It also means that when you are going to manage 34:51.040 --> 34:53.040 your own extension, your own stats, 34:53.040 --> 34:55.040 you have this observable part, 34:55.040 --> 34:57.040 which is local to the backend, 34:57.040 --> 35:02.040 and you have this share stats on memory. 35:02.040 --> 35:04.040 And the observable part, 35:04.040 --> 35:05.040 you can work on it. 35:05.040 --> 35:07.040 Maybe you are communicating information, 35:07.040 --> 35:10.040 and you can process it before merging with the main memory. 35:10.040 --> 35:12.040 That's a lot of thing to do in this era. 35:17.040 --> 35:25.040 One point here also. 35:25.040 --> 35:30.040 The stats in polystyrene are managed by just three by 2D. 35:30.040 --> 35:33.040 The Tabas of ID, object is on T4. 35:33.040 --> 35:35.040 So if it's a new valid one, 35:35.040 --> 35:38.040 it means that the stats are shared on the cluster. 35:38.040 --> 35:40.040 If there is a Tabas ID defined, 35:40.040 --> 35:43.040 this is only for the Tabas in this database. 35:44.040 --> 35:46.040 And then there is an object, 35:46.040 --> 35:47.040 OID. 35:47.040 --> 35:48.040 Before it was an ID, 35:48.040 --> 35:51.040 because before we had stats only for object impossible. 35:51.040 --> 35:54.040 But today we have also stats for backends. 35:54.040 --> 35:56.040 And backends do not have an object ID, 35:56.040 --> 35:57.040 they have a PID. 35:57.040 --> 35:59.040 So the object ID here can be, 35:59.040 --> 36:00.040 for example, PID, 36:00.040 --> 36:02.040 or real OID, 36:02.040 --> 36:04.040 or whatever you want to do with it. 36:07.040 --> 36:08.040 On the other point, 36:08.040 --> 36:10.040 that for the scale is then going to use 36:10.040 --> 36:11.040 key of these structures, 36:11.040 --> 36:12.040 the global structure, 36:12.040 --> 36:14.040 to reference your stats, 36:14.040 --> 36:16.040 and to access it quickly. 36:16.040 --> 36:18.040 So what I'm describing here is correct, 36:18.040 --> 36:19.040 it's true. 36:19.040 --> 36:20.040 But it's going for a bit of change 36:20.040 --> 36:22.040 a bit before physical 18 is released. 36:22.040 --> 36:23.040 Okay. 36:23.040 --> 36:25.040 But more or less, the ID and the concept is true, 36:25.040 --> 36:26.040 we may. 36:29.040 --> 36:30.040 So finally, 36:30.040 --> 36:33.040 stats for extension. 36:33.040 --> 36:36.040 This is an example based on a Pax, 36:36.040 --> 36:39.040 Pax is a particular administrative statistics. 36:39.040 --> 36:43.040 It's just simple extension for physical 18, 36:43.040 --> 36:45.040 which is going to provide new conters, 36:45.040 --> 36:48.040 means you go to Instagram and so on. 36:48.040 --> 36:50.040 So instead of writing your own extension 36:50.040 --> 36:52.040 to manage the new extagram or gore, 36:52.040 --> 36:54.040 you may use Pax, 36:54.040 --> 36:56.040 at least to get the Idders. 36:56.040 --> 36:58.040 And so you already get all the API 36:58.040 --> 36:59.040 to create misuse, 36:59.040 --> 37:01.040 go to conters and Instagrams, 37:01.040 --> 37:03.040 and that you can use in your own extension 37:03.040 --> 37:06.040 to do your own resipe with that. 37:07.040 --> 37:09.040 So here's a simple example with a misuse, 37:09.040 --> 37:12.040 so we are just creating a new stat. 37:12.040 --> 37:14.040 We call it misuse, 37:14.040 --> 37:17.040 and you see the first part is a local one, 37:17.040 --> 37:19.040 so you are defined everything I want. 37:19.040 --> 37:22.040 I just have a single pointer called misuse. 37:22.040 --> 37:25.040 And there is a boiler placed structure for all misuse. 37:25.040 --> 37:28.040 So this one is going to be the placeholder, 37:28.040 --> 37:30.040 I will say, for the first one. 37:30.040 --> 37:31.040 Okay. 37:31.040 --> 37:32.040 If you, 37:32.040 --> 37:35.040 this is a simple result. 37:35.040 --> 37:37.040 If you want more conters, 37:37.040 --> 37:39.040 you can add more conters in the first structure, 37:39.040 --> 37:41.040 and you go. 37:43.040 --> 37:44.040 Then, 37:44.040 --> 37:46.040 if you have defined those two structures, 37:46.040 --> 37:47.040 or you misuse, 37:47.040 --> 37:51.040 you need to register the system into possessual. 37:51.040 --> 37:52.040 Okay. 37:52.040 --> 37:53.040 So for that, 37:53.040 --> 37:56.040 you will need to load it in child memory, 37:56.040 --> 37:57.040 library, and so on. 37:57.040 --> 37:59.040 But more important in the code, 37:59.040 --> 38:02.040 you will have this Pitchy stat kind of 38:02.040 --> 38:04.040 for structure which is provided by possessual, 38:04.040 --> 38:06.040 with strict number of members, 38:06.040 --> 38:08.040 and with members to declare 38:08.040 --> 38:12.040 the name of your new stat. 38:12.040 --> 38:16.040 And if it's a viable number of objects, 38:16.040 --> 38:18.040 or fixed number of objects, 38:18.040 --> 38:19.040 so it's visible, 38:19.040 --> 38:20.040 available, 38:20.040 --> 38:21.040 because we manage the memory, 38:21.040 --> 38:23.040 and increase memory as needed. 38:25.040 --> 38:29.040 You can define if it's a course, 38:30.040 --> 38:34.040 as all the tabases or pair the tabases, 38:34.040 --> 38:37.040 which make me realize that I said something wrong before, 38:37.040 --> 38:38.040 because the database of the ID, 38:38.040 --> 38:40.040 which we saw just before, 38:40.040 --> 38:46.040 can be invalid also in a non-share database. 38:46.040 --> 38:49.040 And something which is very recent, 38:49.040 --> 38:52.040 you can also define if you want to statistic to be right into disk, 38:52.040 --> 38:54.040 because maybe it's not interesting, 38:54.040 --> 38:57.040 maybe you want to start only for live activity, 38:57.040 --> 38:59.040 because there is no point in trying to flesh that to disk. 38:59.040 --> 39:01.040 So in order to not aflashing at the end, 39:01.040 --> 39:03.040 when you restart or stop at a square, 39:03.040 --> 39:05.040 you can define right to find the force, 39:05.040 --> 39:07.040 and here you are. 39:09.040 --> 39:11.040 So I did not list all the members, 39:11.040 --> 39:14.040 but what I really was the most important to understand, 39:14.040 --> 39:17.040 what is the simplest way and the easiest way 39:17.040 --> 39:19.040 to get your own extension. 39:20.040 --> 39:22.040 So as usual, you just need to inform 39:22.040 --> 39:24.040 about the sizing of your data. 39:24.040 --> 39:27.040 You created a new institute with members with a new view, 39:27.040 --> 39:30.040 so what is the size of his structure? 39:30.040 --> 39:34.040 And what is the offset to this stats member? 39:34.040 --> 39:37.040 Remember, the stats number is on the shelves, 39:37.040 --> 39:38.040 the global structure, 39:38.040 --> 39:42.040 and this is what the school is going to use to find your stats. 39:42.040 --> 39:45.040 So you just give it the offset to the member, 39:45.040 --> 39:47.040 and boom, you go to it. 39:50.040 --> 39:53.040 And then you have some callback that you can define. 39:53.040 --> 39:55.040 For example, reset time stamp. 39:55.040 --> 39:57.040 By default, on the stats, you do not have a reset time stamp. 39:57.040 --> 39:59.040 This is global information. 39:59.040 --> 40:01.040 When did you last reset to the stats? 40:01.040 --> 40:03.040 But if you want to be able to reset statistics 40:03.040 --> 40:06.040 for each row of each stats, 40:06.040 --> 40:07.040 you can do that, 40:07.040 --> 40:09.040 and you can add a new member in the structure 40:09.040 --> 40:11.040 to keep information about the stats, 40:11.040 --> 40:12.040 timestamp, 40:12.040 --> 40:14.040 and also provide the callback functions 40:14.040 --> 40:17.040 to inform political what he must do to reset the stats. 40:17.040 --> 40:19.040 Not to reset pattern. 40:19.040 --> 40:22.040 What he must do to register 40:22.040 --> 40:25.040 the last time you reset statistics. 40:27.040 --> 40:30.040 There are two members, two say here as name, 40:30.040 --> 40:34.040 and formula is name, which are not interesting for me, 40:34.040 --> 40:35.040 yet at the moment, 40:35.040 --> 40:38.040 but which are using political further applications lot. 40:38.040 --> 40:40.040 You know, when you have a application certain political, 40:40.040 --> 40:41.040 you connect the back end, 40:41.040 --> 40:43.040 but this application is not as a name. 40:43.040 --> 40:45.040 It does not have an object idea whatsoever. 40:45.040 --> 40:48.040 So the way to follow up here is to use the name, 40:48.040 --> 40:51.040 and so we have for callback functions, 40:51.040 --> 40:55.040 we are able to move from string to a key to a string, 40:55.040 --> 40:57.040 and which is used for application set, 40:57.040 --> 40:59.040 and maybe for future extension. 41:04.040 --> 41:07.040 The panic size here is also interesting 41:07.040 --> 41:10.040 because if panic size is defined as zero, 41:10.040 --> 41:11.040 there is no panic entries. 41:11.040 --> 41:13.040 So you're walking only on the main memory. 41:13.040 --> 41:15.040 There is no local memory. 41:15.040 --> 41:17.040 If you define the panic size, 41:17.040 --> 41:19.040 you can have just one structure, 41:19.040 --> 41:21.040 but you can do whatever you want. 41:21.040 --> 41:23.040 This is to manage your panic list, 41:23.040 --> 41:24.040 and your panic statistic, 41:24.040 --> 41:26.040 that you want to push later. 41:26.040 --> 41:28.040 And then you define a function. 41:28.040 --> 41:30.040 It has callback to flesh. 41:30.040 --> 41:34.040 It means to merge the statistic from your local back end to the main memory. 41:34.040 --> 41:36.040 It looks like it's a bit boring, 41:36.040 --> 41:41.040 but if you just write those few callback functions, 41:41.040 --> 41:43.040 your system is going to work. 41:44.040 --> 41:48.040 For instance, this is what you need to have in the call. 41:48.040 --> 41:51.040 You need to define. 41:51.040 --> 41:54.040 You need more, 41:54.040 --> 41:57.040 firstly, when the protocol is starting, 41:57.040 --> 41:59.040 it's going to load module coming from chat, 41:59.040 --> 42:01.040 but load device parameter, 42:01.040 --> 42:05.040 and so you add your extension or your module to this list, 42:05.040 --> 42:08.040 and so it's going to execute PG in it of your module. 42:08.040 --> 42:10.040 When it's going to execute PG in it, 42:10.040 --> 42:12.040 you can register your new kind of statistics. 42:12.040 --> 42:14.040 Okay? Easy. 42:14.040 --> 42:17.040 But only doing startup. 42:17.040 --> 42:19.040 And then you just have to define, 42:19.040 --> 42:22.040 okay, when I want to flush statistic from my back end to main memory, 42:22.040 --> 42:24.040 what do I do? 42:24.040 --> 42:26.040 I get the panic entries, 42:26.040 --> 42:28.040 and I get the style start, 42:28.040 --> 42:29.040 and I say, 42:29.040 --> 42:32.040 okay, the shell statistics of my measure, 42:32.040 --> 42:35.040 it just plus the local measure. 42:35.040 --> 42:38.040 So if I have 10 in main memory, 42:38.040 --> 42:40.040 and I'm already at 5, 42:40.040 --> 42:42.040 then it's 5, 15, 42:42.040 --> 42:44.040 and I have that measured memory. 42:44.040 --> 42:45.040 That's all. 42:45.040 --> 42:48.040 But if you want to do something else, 42:48.040 --> 42:50.040 you can do something else. 42:50.040 --> 42:52.040 If you want to do at the same time, 42:52.040 --> 42:56.040 a mean, a max, average, 42:56.040 --> 42:59.040 rates, ratio, whatsoever you want to do, 42:59.040 --> 43:02.040 when it's pushing, you can even say, 43:02.040 --> 43:04.040 okay, but here, example, 43:04.040 --> 43:08.040 I increment in my value 10 times, 43:08.040 --> 43:10.040 in 20 minutes, 43:10.040 --> 43:13.040 maybe I can, I want to register that, 43:13.040 --> 43:14.040 I can, 43:14.040 --> 43:16.040 I can use that to produce aggregate of 43:16.040 --> 43:18.040 frequency rates in my main memory. 43:22.040 --> 43:25.040 Reset times times is very easy. 43:25.040 --> 43:27.040 There's a callback function, 43:27.040 --> 43:29.040 and so you just say, okay, 43:29.040 --> 43:31.040 what is the member in mass structure, 43:31.040 --> 43:34.040 which is holding the timestamp of my last visit. 43:34.040 --> 43:35.040 That's all. 43:38.040 --> 43:43.040 And now we need to use it. 43:43.040 --> 43:45.040 So in this first example, 43:45.040 --> 43:46.040 this is an increment, 43:46.040 --> 43:48.040 which is not only on the main memory, 43:48.040 --> 43:49.040 on child memory directly. 43:49.040 --> 43:51.040 We bypass the local memory here, 43:51.040 --> 43:55.040 because we are directly asking to get a lock 43:55.040 --> 43:58.040 on the entry ref in the child memory. 43:58.040 --> 44:00.040 And so we lock it, 44:00.040 --> 44:02.040 and so we can work with it. 44:02.040 --> 44:04.040 So this update here, 44:04.040 --> 44:05.040 update the measure, 44:05.040 --> 44:07.040 is done directly in the child memory, 44:07.040 --> 44:09.040 and visible for everyone, 44:09.040 --> 44:11.040 as soon as this function has ended. 44:11.040 --> 44:13.040 Okay? 44:13.040 --> 44:14.040 Why? 44:14.040 --> 44:16.040 And so you notice here, 44:16.040 --> 44:19.040 that BGDL export which allows 44:19.040 --> 44:21.040 to get this function exported to another model, 44:21.040 --> 44:22.040 another extension, 44:22.040 --> 44:24.040 which means that if you want to use measure 44:24.040 --> 44:25.040 and increment to measure, 44:25.040 --> 44:27.040 instead of writing again this new 44:27.040 --> 44:28.040 this function, 44:28.040 --> 44:31.040 you can just use it in your own code. 44:31.040 --> 44:37.040 But if you want to go in the pending entry, 44:37.040 --> 44:39.040 it means that you're going to increase the 44:39.040 --> 44:40.040 measurement not globally, 44:40.040 --> 44:41.040 not on child memory directly, 44:41.040 --> 44:42.040 but on the local back end. 44:42.040 --> 44:45.040 There is also a function for ID by processual, 44:45.040 --> 44:47.040 which is going to prepare for you, 44:47.040 --> 44:50.040 room for this data. 44:50.040 --> 44:52.040 So this function is going to manage, 44:52.040 --> 44:53.040 there's a structure exist, 44:53.040 --> 44:55.040 as we declare the size is okay, 44:55.040 --> 44:57.040 and so you can start using it. 44:58.040 --> 45:00.040 And then, 45:00.040 --> 45:02.040 just after here you recover, 45:02.040 --> 45:04.040 you get your entry, you're planning entry, 45:04.040 --> 45:07.040 and so you can do a measure plus plus, 45:07.040 --> 45:10.040 so you can manage your but only locally here. 45:10.040 --> 45:11.040 So it's visible locally, 45:11.040 --> 45:14.040 it's not visible in the main memory. 45:16.040 --> 45:17.040 And then, 45:17.040 --> 45:18.040 it gets me, 45:18.040 --> 45:20.040 because you want to get the value at some point. 45:20.040 --> 45:21.040 And see, 45:21.040 --> 45:23.040 it's very similar also, 45:23.040 --> 45:25.040 except that I decided to add a function 45:25.040 --> 45:26.040 to say, 45:26.040 --> 45:30.040 I want the local value of the global value. 45:30.040 --> 45:31.040 And so bad on that, 45:31.040 --> 45:34.040 you get the local also remote value. 45:34.040 --> 45:35.040 So it means at the same time, 45:35.040 --> 45:36.040 you have only one measure, 45:36.040 --> 45:40.040 but already has two points of existence 45:40.040 --> 45:42.040 and two distinct value possible. 45:45.040 --> 45:47.040 If you go on a scale, 45:47.040 --> 45:49.040 you have that. 45:49.040 --> 45:52.040 Notice that I do a big in at the beginning, 45:52.040 --> 45:54.040 okay, this is transaction. 45:54.040 --> 45:55.040 So an action has a point here, 45:55.040 --> 45:58.040 because when you are inside transaction, 45:58.040 --> 45:59.040 by default, 45:59.040 --> 46:02.040 it's not going to snapshot and two pattern. 46:02.040 --> 46:04.040 By default, it's not going to flash your start. 46:04.040 --> 46:05.040 You're in transaction, 46:05.040 --> 46:08.040 and often the stats are flush in memory 46:08.040 --> 46:10.040 at the end of the transaction. 46:10.040 --> 46:11.040 So I put a big in, 46:11.040 --> 46:13.040 and I increment for them, 46:13.040 --> 46:15.040 this is a new pointer. 46:15.040 --> 46:16.040 And so I have one. 46:16.040 --> 46:17.040 You see, 46:17.040 --> 46:18.040 if I select, 46:18.040 --> 46:19.040 get me a pending true, 46:19.040 --> 46:20.040 I get this one, 46:20.040 --> 46:21.040 which is a local value, 46:21.040 --> 46:23.040 I get zero on the shell start. 46:24.040 --> 46:26.040 But then, 46:26.040 --> 46:29.040 I come and my transaction, 46:29.040 --> 46:31.040 so it's going to do a commit or a walk back, 46:31.040 --> 46:34.040 according to the previous mistake. 46:34.040 --> 46:35.040 So here's going to commit. 46:35.040 --> 46:37.040 And when it has committed, 46:37.040 --> 46:40.040 my get me from the main memory, 46:40.040 --> 46:42.040 because then it's forced, 46:42.040 --> 46:44.040 as been updated, 46:44.040 --> 46:48.040 and my local memory has been flushed, 46:48.040 --> 46:50.040 and it's zero today. 46:50.040 --> 46:51.040 Okay? 46:51.040 --> 46:53.040 Very easy to get one measure, 46:53.040 --> 46:55.040 with two possible pointers inside, 46:55.040 --> 46:57.040 between my local activity, 46:57.040 --> 46:59.040 and the main memory. 47:02.040 --> 47:04.040 If you want to go more on the topics, 47:04.040 --> 47:06.040 you can improve your structure, 47:06.040 --> 47:07.040 and you get a measure, 47:07.040 --> 47:09.040 is your main max, 47:09.040 --> 47:12.040 right, everything. 47:12.040 --> 47:14.040 If you want to walk with these programs, 47:14.040 --> 47:16.040 not a problem. 47:17.040 --> 47:21.040 Here, I use a fixed number of, 47:21.040 --> 47:24.040 an array with a fixed number. 47:24.040 --> 47:25.040 And honestly, 47:25.040 --> 47:28.040 I'm not sure that we can use a fixed number, 47:28.040 --> 47:30.040 with this part, 47:30.040 --> 47:32.040 because what's currently then going to increase the size, 47:32.040 --> 47:34.040 and it does not know about how much. 47:34.040 --> 47:36.040 So, except that it's for M, 47:36.040 --> 47:40.040 it's very easy to build a histogram of how many value you want. 47:40.040 --> 47:42.040 And this diagram will be on your local memory, 47:42.040 --> 47:43.040 or in the main memory. 47:43.040 --> 47:45.040 And again, you can walk with both sides. 47:45.040 --> 47:48.040 Consider that the local memory is an observable pointer, 47:48.040 --> 47:50.040 an observable matrix, 47:50.040 --> 47:53.040 and the main memory is a, 47:53.040 --> 47:55.040 non-observable pointer. 48:02.040 --> 48:04.040 When we saw an angle the slide, 48:04.040 --> 48:07.040 but being there, 48:07.040 --> 48:09.040 I had some few questions, 48:09.040 --> 48:12.040 because I was not able to list all the stats. 48:12.040 --> 48:15.040 And if I create a number of contours with my new extension, 48:15.040 --> 48:17.040 there is no way for me to list them. 48:17.040 --> 48:18.040 I cannot know. 48:18.040 --> 48:20.040 So, I must, which is the name of the contours. 48:20.040 --> 48:23.040 So, I must, which is something to find my contours again. 48:23.040 --> 48:26.040 So, maybe there is some improvement to do in a API 48:26.040 --> 48:28.040 to be able to list that from extension. 48:28.040 --> 48:34.040 So, you can check what you have already added to the main memory. 48:34.040 --> 48:39.040 It will be very interesting to get a better control of when you flash the stats. 48:39.040 --> 48:42.040 Because at the moment, it's a little bit fuzzy. 48:42.040 --> 48:45.040 I mean, this is done by transaction, by time out of the things. 48:45.040 --> 48:51.040 But, few functions have been added to be able to let you decide 48:51.040 --> 48:54.040 more precisely when not to do it, 48:54.040 --> 48:57.040 or bypass the pending list and have your own pending list 48:57.040 --> 48:59.040 that you manage on your own, 48:59.040 --> 49:01.040 but still a little bit are. 49:01.040 --> 49:04.040 And I believe it will be very interesting that you can quickly 49:04.040 --> 49:07.040 integrate extension statistics, 49:07.040 --> 49:11.040 and make sure that extension developer can decide when the stats 49:11.040 --> 49:13.040 should be flashed, merge, and so on. 49:13.040 --> 49:16.040 And not let for the studio decide for that. 49:16.040 --> 49:17.040 Because there will be a fun, 49:17.040 --> 49:19.040 I think that if you have extension statistics, 49:19.040 --> 49:22.040 it may be more of an application point of view 49:22.040 --> 49:25.040 and it's not the same expectation from the other stats. 49:30.040 --> 49:32.040 Okay, so, if I'm not too late, 49:32.040 --> 49:35.040 we have some time for questions, maybe. 49:35.040 --> 49:36.040 Maybe not. 49:36.040 --> 49:38.040 Probably just one question. 49:44.040 --> 49:46.040 Very, very intense talk. 49:46.040 --> 49:48.040 Very technical. 49:48.040 --> 49:49.040 Yes. 49:49.040 --> 49:50.040 I think we'll judge them. 49:50.040 --> 49:51.040 Yeah, of course. 49:51.040 --> 49:53.040 We have to do it. 49:53.040 --> 49:54.040 Any question?