WEBVTT 00:00.000 --> 00:14.160 So, I will be there presenting how at television we are using a differ for embedded video 00:14.160 --> 00:15.160 system. 00:15.160 --> 00:23.600 So, bit background at me, I am a contractor mostly working with television which is a design 00:23.600 --> 00:28.880 service company and we do we build embedded hardware for anyone who has got projects to 00:28.880 --> 00:35.880 build but not all the skills from PCB to firmware to FPGA programming to do that and 00:35.880 --> 00:40.600 the FPVitas and we have them building the real products hopefully to put them on a market 00:40.600 --> 00:43.600 someday you say, if you go this far. 00:43.600 --> 00:51.320 It's a good illustration of what I typically do and they are picking video system, various 00:51.320 --> 00:58.040 conditions with the same laptop on a different packet but as you hear, I am curious about 00:58.040 --> 01:02.240 a lot of the means and I scratch the surface of everything, probably a lot of you will have 01:02.240 --> 01:07.400 a very good experience in one domain which will be absolutely enabled to understand and one 01:07.400 --> 01:13.000 of my goal is somehow to be able to interface the people who are experts in the domain with 01:13.000 --> 01:19.800 their actual art they have to do but on smaller system, on embedded system which are more 01:19.800 --> 01:25.240 challenging than self because you have to do firmware was there is an application programming 01:25.240 --> 01:30.840 and we will see how we can start to approach this kind of challenging system to build 01:30.840 --> 01:32.840 video out of it. 01:32.840 --> 01:39.080 So that is because very familiar examples that everybody might have in mind, kind of promising 01:39.080 --> 01:45.960 my system, very typical but that's not a complete video system you have the parts that 01:45.960 --> 01:52.880 you see here but the images coming from somewhere also and if you have to pick the whole 01:52.880 --> 01:58.080 system and to them from the images produced to when it is arrived, that is a lot of steps 01:58.080 --> 02:02.560 if you try to catch them in categories, you will have a professional video camera like 02:02.560 --> 02:07.360 the one that is filming me and the video editing software sometimes that happens in a 02:07.360 --> 02:12.400 well time when the video stream is coming on, a distribution network that brings the image 02:12.400 --> 02:21.280 to where it is used or watched and finally the home cinema that we are that we are so relatively 02:21.280 --> 02:29.120 to embedded system, we could say that the budget is rather limited for the kind of equipment, 02:29.120 --> 02:33.680 for instance the camera budget if you can make better camera road to make better movies, 02:33.680 --> 02:38.720 the movie company will buy them so we can go out almost as far as we want in the quality 02:38.720 --> 02:44.960 and we have very nice image sensor in this device if you do it in software I guess the 02:44.960 --> 02:50.480 pixel that a center is for rendering, express the lack of risk to tell the budget constraint 02:50.480 --> 02:56.720 for editing software resources, in some case of course not every system is like that but we can 02:56.720 --> 03:05.840 also go very far in a distribution, there is gigabits, internet networks, as high resolution 03:05.840 --> 03:11.920 video transmitted on demand, as a scale of embedded system that's not the kind of low 03:11.920 --> 03:17.920 constraint resource that we are used for and the home cinema is kind of a comfortable system 03:17.920 --> 03:24.240 to watch video but if you walk on every video system you might have to shrink all that 03:24.880 --> 03:31.120 into something like a baby phone that's not the exact same kind of a budget unlimited budget 03:31.120 --> 03:37.280 constraint that you have, you will have to put a price on it, you will have to get people to buy 03:37.280 --> 03:42.640 it and if it was costing 10 times the price which is only a certain cheap for high quality 03:42.640 --> 03:49.840 camera people the product will not exist altogether and you find all of these elements inside 03:49.840 --> 03:56.320 you have a huge cost budget processing budget because you cannot buy expensive part into it 03:57.360 --> 04:03.120 also a huge power budget so if you get powerful parts, powerful CPU side you will run 04:03.120 --> 04:08.000 out of power for battery device and you have time budget because sometimes things need to be 04:08.000 --> 04:13.440 very low latency and reactive so if you integrate some processing in the middle this will 04:13.440 --> 04:20.320 affect the performance of the system that suggests we can only walk at low resolution but 04:20.320 --> 04:27.200 that's not entirely true because if we look at the professional video camera there is a system 04:27.200 --> 04:31.920 running inside the camera as well as there we need some software inside the camera itself 04:32.880 --> 04:38.720 which might be a complete full stage line system or depending on the use case it might be 04:38.720 --> 04:44.000 a smaller lower power system because it can arise sometimes even higher quality one 04:44.000 --> 04:50.240 battery operated and everywhere you can shrink the system to consume fewer power then it's a good 04:50.240 --> 04:59.120 thing to take so it's possible to have a smaller system in the video device system so it's not always 04:59.200 --> 05:04.880 low end so further embedded devices we think about some very cheap things that we distribute 05:04.880 --> 05:13.760 on the market for sense but all sort of system high and low end are using our prevent embedded 05:13.760 --> 05:21.200 world that's mostly a question of which scale we build things if you build something that's 05:21.200 --> 05:26.480 start a powerful system it's a powerful device but you still have an embedded system somewhere 05:26.560 --> 05:30.480 and you look very close at the resources and that's probably what defines the 05:30.480 --> 05:36.400 MLED systems close to the hardware close to the phenomenon that you use 05:38.560 --> 05:44.080 so what if you want to implement a video camera we could see go with something simple because 05:44.080 --> 05:51.120 Raspberry Pi computer and plug USB camera to it and bang we have a video system and embedded one 05:51.200 --> 05:57.680 low power too that's true that works with embedded system the idea that we are looking 05:57.680 --> 06:02.000 inside the USB camera for instance what is inside the device where it doesn't 06:02.000 --> 06:07.920 using a device if you look what implements the device this kind of operating system we are 06:07.920 --> 06:13.120 observing and one possible operating system for doing that is differ which is 06:14.080 --> 06:21.520 a very familiar to Linux API but it's a completely trimmed down system and it works on 06:21.520 --> 06:29.520 much smaller chips that's an example of a product we actually build USB camera so that's 06:29.520 --> 06:37.760 a very small module that's used in terms of this three camera and we do not find a lot of 06:37.760 --> 06:42.720 Linux simple good work computer of this size you can actually see by because it's scaled up but 06:42.720 --> 06:49.840 it's very tiny so we have to find video systems that can implement a complete video pipeline 06:49.840 --> 06:56.000 but smaller but on that one of smaller chips on what can around Linux and then we have to 06:56.000 --> 07:03.200 reinvent the complete video system for such smaller chips that's an example of how you can build 07:04.160 --> 07:10.400 camera device out of such a small board you can connect it to a USB connector and you have 07:10.400 --> 07:15.360 camera connector and that's just to be a lot of a small USB camera for instance 07:17.440 --> 07:21.280 so you could even to click Raspberry Pi Team to not use B camera that will work 07:22.400 --> 07:26.880 or a small embedded camera and they'll actually company who do this you will have a 07:26.880 --> 07:32.480 poor budget that's already by concentrating quite a bit of power it's if you want to have 07:32.480 --> 07:39.120 battery operated device you start to really want to shrink shrink shrink the power operation 07:39.120 --> 07:43.600 the power performance is good but that depends what you mean by my good that you will always 07:43.600 --> 07:48.960 have some device that is a little bit more the one typical of the shift device requires 07:48.960 --> 07:54.080 and the cost latency will also be a limitation in some case depending on what you do 07:54.080 --> 07:57.840 but it has so many to three possible that's how do come is that do the camera model that's 07:57.840 --> 08:02.480 Raspberry Pi and you can play camera to it and that's very good what camera is if you wish to 08:02.480 --> 08:10.000 it's not using a Linux system that works so every video system is not about the size of the device 08:10.000 --> 08:16.560 itself it would say shrink it shrinks the constraints but there's also some very large video system 08:16.560 --> 08:22.320 which has a whole thing called very large to this cup and chill it and it's in this very large 08:22.320 --> 08:28.160 it's also a good opportunity to be able to jump inside of an embedded device embedded because 08:28.160 --> 08:35.040 the computer system is part of the building or device that just an opportunity to to picture yourself 08:35.040 --> 08:40.080 what can be done in the device kind of a crazy machinery all of which we could be even to be 08:40.080 --> 08:46.320 shrink in the time of camera I have no idea how the revolutionary scope works but 08:46.880 --> 08:55.280 just good illustration because we can imagine how this all different aspects of video system come 08:55.280 --> 09:02.480 together you will need some kind of networking for instance in different studies net you have some 09:02.480 --> 09:08.560 motor control for controlling the aperture, the auto focus, etc we have flaps in the embedded 09:08.560 --> 09:15.040 camera aperture to controls the amount of light that come into the system you have all sort of 09:15.040 --> 09:20.720 indicate all sense of the building interface connectivity for instance with canvas or all the kind of 09:20.720 --> 09:25.920 system of course on the camera you not have such a large computer system which I have no idea how 09:25.920 --> 09:32.720 works but you will have some idea of for instance instead of a large computer work for motor 09:32.720 --> 09:37.760 or for opening doors you will have a voice call motor for autofocus as dedicated just to 09:37.760 --> 09:43.760 a quickly reacting to focus change and you will probably have a small tip that controls the motor 09:43.760 --> 09:47.760 and I have ice policy communication with that chip for instance in differ you can use 09:47.760 --> 09:53.360 ice question library that's dedicated for this kind of use case and you will want to 09:53.360 --> 09:57.680 pour off the device as much as possible when you're not using it I'm quickly pour it back on 09:57.680 --> 10:04.480 so this poor management integrated into the function of the whole device for instance in different 10:04.480 --> 10:14.400 there's a poor management system that's dedicated to turn seem on and off so we are that's a 10:14.400 --> 10:19.200 view of how to implement a video system that you will typically run with V42 which is excellent 10:19.200 --> 10:25.440 API for being a super hardware support but unfortunately in some cases some video device cannot 10:25.440 --> 10:31.280 be built with a complete powerful device and you have to use an autos and one such autos which 10:31.280 --> 10:38.480 you try to use for video is differ so what are currently supported on differ if you can currently 10:38.480 --> 10:45.520 just load them a different autos and use them for video devices just all of the people what 10:45.520 --> 10:52.480 that might look like physical if what depth can take are there around the way that the goal 10:52.480 --> 10:58.000 is always we suffer larger devices and we try to shrink them and shrink and see whole small 10:58.000 --> 11:08.960 we can get with an in time of size or cost or poor constraints so I am ex artis kind of the bigger one 11:08.960 --> 11:15.600 it's a gigahertz CPU that cannot run Linux not so many models that kind if good to have it's 11:15.600 --> 11:22.400 good to have fast video CPU in device because when you walk this buffer if you flush the buffer 11:22.400 --> 11:27.280 more frequently that means that you can get the same that that's what put as you have large 11:27.280 --> 11:32.960 buffers that you flush once in a while so if you can go faster you can have smaller buffers 11:32.960 --> 11:38.640 and in turn it means that you can have fewer RAM and you can eventually omits the RAM chip 11:38.640 --> 11:44.640 so you can save power by removing parts of your system also you can use a built in RAM in some 11:44.640 --> 11:48.880 in some cases it's not so that's kind of kind of large but that's just because there's a lot of 11:48.880 --> 11:54.240 different things on it and it's used for all the debugging VPN image sensors for constantly in 11:54.240 --> 12:00.400 easier since NXP team working on it for instance and you can have you can have internet on all 12:00.400 --> 12:09.280 sort of a of a city on board and together all the 92 of the resources you can see that there's 12:09.280 --> 12:14.560 lot of trouble coming in maybe you can have a display out for instance if you have a preview display 12:14.560 --> 12:18.720 on the camera what you have something directly in the hardware that avoids you to between the 12:18.720 --> 12:25.760 Nazis with software which will be too much for even for your health system you have an external 12:25.760 --> 12:31.120 interface such as USB 2 or ethernet you can kind of have the same range of properties for the 12:31.120 --> 12:36.720 attack that comes in and the data that comes out all of my data the same and very fast CPU 12:36.720 --> 12:42.320 course typically when you have two CPU course one will be for control one will be for the data 12:42.320 --> 12:46.880 number crunching so that you can use as much as the number crunching and get all the control 12:46.880 --> 12:54.320 all of the powerhouse of the board so that's the system we build and it's going to be a 12:54.320 --> 12:59.280 particular system to work with because it's FTA so you customize the contents of the chip 12:59.280 --> 13:04.480 you choose how the CPU looks like you should have the video accelerators look like and it's 13:04.480 --> 13:09.520 up to you to provide this call this is why television is also designed to save this part 13:09.520 --> 13:16.160 now because it's kind of challenging to work with FGA so it takes more time so it's a resources 13:16.160 --> 13:22.560 with a with a company to help anyone who wants to build system also for FGA and one thing that's 13:22.560 --> 13:31.520 kind of strange is the use with earlier I mean that's super nice but what 18-gahead CPU 13:31.520 --> 13:37.840 for so many MP lines and so much throughput that doesn't make sense you cannot process the video 13:38.800 --> 13:44.000 so that's a good example of embedded system that can be totally disbalanced they can be very 13:44.000 --> 13:50.160 specialized in transmitting from one side to the other and if there is other acceleration to go from 13:50.160 --> 13:55.440 one side to two the other well you do not have to pay so much about the CPU because it's not 13:55.440 --> 14:01.600 involved in the actual operation of the device so in the end the CPU is only there in the corner 14:01.600 --> 14:07.120 to keep the control operation going you do not need a large complex system like legacy if you just 14:07.120 --> 14:12.000 want that you will have to make the system much larger and what are the results would be used for 14:12.000 --> 14:18.000 a small control algorithm you do not need such a small at all that's why we are that's the 14:18.000 --> 14:24.000 reason for which we are using a different hour hour ecosystem and in other example we start to 14:24.000 --> 14:30.000 shrink a little bit as the other boss from NXP different as a slightly smaller CPU and you will find 14:30.000 --> 14:35.760 some kind of interesting peripherals that you usually find on larger board like ATMX ATM plus 14:37.120 --> 14:41.840 and like a neural processing and trying to do AI inferencing if you want to detect something 14:41.840 --> 14:47.200 on the image on the device directly and react to it very fast for instance so you will have 14:47.200 --> 14:52.960 this kind of wall which is relatively compact also much more than the one we saw earlier which is 14:52.960 --> 14:59.120 one internet interface for instance only gets like the result gets shrinking a bit smaller the 14:59.120 --> 15:05.600 video device less capable but also much lower power and you have interesting poticle special 15:05.600 --> 15:12.640 purpose course which I have a neural inference core insights if you want to collect the image frames 15:12.640 --> 15:17.360 pass it to a detection and giant and tell you that with someone on the system for instance 15:17.360 --> 15:22.160 you will want to detect if someone is walking in front of a device to wake up the device 15:22.160 --> 15:26.240 and save power where there is no one in front you can you have to detect it there is something that 15:26.240 --> 15:31.920 looks like a person and to do that you cannot communicate with internet ask get the result back you 15:31.920 --> 15:37.440 have to do it on the device really embedded and if it's a feature that is to reduce the amount of 15:37.440 --> 15:44.720 power the system itself cannot consume a lot of power and you have to try to find ways to shrink 15:44.720 --> 15:49.840 the system and one way is to use fewer RAM and then switch to embedded operating system that 15:49.840 --> 15:59.840 brings you to reduce the power footprint also board this time very physically small you will 15:59.840 --> 16:05.920 have a very compact board with a GPO pin for controlling things around it with a camera inside 16:05.920 --> 16:12.880 with a Wi-Fi with an IMU for detecting the orientation of the board so you start to have a lot of 16:12.880 --> 16:18.160 hardware things how to gadget that permits to associate with the video for instance you can 16:18.160 --> 16:24.960 know the direction which you look if you use the orientation sensor the IMU and you have 16:25.280 --> 16:31.040 some kind of series available on the processing power to do you can do a few video processing some 16:31.040 --> 16:36.880 kind of converting the video trying to convert the pixel formats or things like that that would be 16:36.880 --> 16:42.320 required for instance with USB communication and you have a GPO compression core which is nice because 16:42.320 --> 16:52.720 it permits to reduce the amount of data is transferred to its body and still equivalent but for Wi-Fi 16:52.720 --> 17:00.320 so you have very small inside this time this is a Wi-Fi slightly high power but you can really get 17:00.320 --> 17:07.200 compact device that will communicate well as using the first and this one is also supported board 17:07.760 --> 17:12.960 you can get some kind of interfaces this time no USB but alternative interface and you 17:12.960 --> 17:18.800 still have this kind of same order of magnitude of between what the image can do and what the data 17:18.800 --> 17:24.480 interface can do so if you have Wi-Fi could eventually transmit full frame of this depending on 17:24.480 --> 17:31.840 how which FPS you like always a kind of matching between the resources of chiplighter so on this 17:31.840 --> 17:36.560 time we want to treat everything away from a board which is not optimized for size but of 17:36.560 --> 17:41.440 you might more for cost you have a chip with almost nothing on it there's a camera connector 17:41.440 --> 17:47.520 than the display connector and that it and you can also customize with the less expensive chips 17:47.520 --> 17:53.680 if you have high volume device you want to with a video system into every watt into every door 17:53.680 --> 17:57.280 it has a lot of durer run you will need something very cost optimized so that you can 17:57.280 --> 18:03.040 actually have a foldable system all the work is it will not be a usable at all and still some 18:03.040 --> 18:12.240 reasonable processing side that's the same as the Nikla so how with fewer processing that's nice 18:12.240 --> 18:18.480 but you leave the comfort of line if programming environment and all of these software stacks 18:18.480 --> 18:24.000 you will have to be reinvented they will have to be redundant different ways so let's look at 18:24.000 --> 18:29.760 what we have now the ever has video API that's fairly low level and that's most of the work 18:29.760 --> 18:35.520 for the application side which also permits to port libraries for vision processing 18:35.520 --> 18:39.200 in particular algorithm you wish to do for instance the QR code detection 18:39.200 --> 18:44.400 well that's one particular thing cool try to pause in one of the available language that you 18:44.400 --> 18:54.000 can run on top of the zipper and so that a video API was introduced by Luke Pula at 2019 and he took 18:54.000 --> 19:00.640 all of the constraint and inspiration from all the others video API put it around and try to adapt 19:00.640 --> 19:07.200 them to the zipper according to what will be pleasant for the existing zipper developers and that 19:07.200 --> 19:14.720 brought something a little bit like this like to picture you have an image sensor that provide 19:14.720 --> 19:21.760 video data to the chip and on the chip you will have a MEP driver that receives a video signals 19:21.760 --> 19:27.200 as I kind of Brian them it to fill them into memory and you have all the memory buffers that are 19:27.200 --> 19:36.480 loaded in with NQ and that taken out full of pixels and you can cycle the buffers continuously 19:36.480 --> 19:42.560 and you like that you like in the conveyor belt that will be filled by some pixels as the buffer 19:42.560 --> 19:47.440 pass in front you load buffers and takes them out that's the NQ and the QR operation of the API 19:49.440 --> 19:56.160 so note that here you have an image sensor every camera every video device need a driver 19:56.160 --> 20:01.680 for the particular image sensor that collects the pixels and it's kind of really hard to get 20:01.680 --> 20:06.080 your hand over the data sheets for the image sensor that's very protected, very private, 20:06.080 --> 20:10.720 for the company who do them as a lot of competitions and it's very good to have access to this 20:10.720 --> 20:20.400 image sensors so if we consider that only the IO interface for the MEP driver not for the 20:20.400 --> 20:27.040 image sensor itself the image sensor is turning data to the hardware as a software only operator 20:27.040 --> 20:33.440 that's a data for the MEP driver that's an example or for you will actually look at the code 20:33.440 --> 20:39.520 you start the device you allocate the buffer and then while true NQ they queue and queue the queue 20:39.520 --> 20:43.840 and then you add to it when you they queue buffer you can do something with a data out of it 20:44.880 --> 20:52.640 for instance process with application so that's the model if we plug it we can use it by first 20:52.640 --> 20:56.560 to plug it to a difference for instance you had dipping in code as you can pass the buffer to a 20:56.560 --> 21:01.840 different device and the device will operate on the data and you'll have to provide new 21:01.840 --> 21:06.640 buffers so that they can receive the converted data and cycle them again like you you will 21:06.640 --> 21:12.400 do in a normal like like you did for the first device and recycle the buffers so that you 21:12.400 --> 21:19.760 don't have to call the allocator every time so that's the NQ and DQ operation of the API 21:19.760 --> 21:26.960 we saw how it's implemented just before and that's that works you can do the exact same way 21:26.960 --> 21:31.760 as before and queue for the first device at the queue and then and queue again to onto the next 21:31.760 --> 21:38.080 device and the queue that works one by one that's that was very parallel you have to wait 21:38.080 --> 21:44.240 at the full image process before you can start to the 7 so you have you can do something similar 21:45.200 --> 21:53.280 with when you unlock all the buffers first you thank you them all first and then on several 21:53.280 --> 21:58.800 thread you'll thank you on the queue on the separate like separate conveyor belts that convey 21:58.800 --> 22:03.040 the data into the devices for instance the first time in thank you the queue and queue the 22:03.040 --> 22:07.520 queue continuously and the salt of pins will also device to load the data in and out 22:09.200 --> 22:13.920 and that will be a second trade for the second profile and answer for every kind of conveyor belt 22:13.920 --> 22:21.280 to have a separate NQ NQ to operate and at the last step you have you can actually use your 22:21.280 --> 22:25.280 JPEG buffer that is ready that is processed you can use policies with a network stack to 22:25.280 --> 22:33.200 send over network or USB for instance so that's the same kind of API but using Paul there's also 22:33.200 --> 22:38.400 pulling API which you can use in video devices so that you do not have so many thread at once which is a 22:38.400 --> 22:45.680 bit a heavy way to use I will let you review that on the slide which will be on the website 22:48.080 --> 22:53.680 so that good parallelism and that's that's a good bedwork for making things a bit more automated 22:53.680 --> 22:58.240 so that you do not have to put your hand onto the device and tell us because it's start to get 22:58.240 --> 23:06.640 low level as this point here every block you can also have controls if you want to tune the 23:06.720 --> 23:11.920 exposure you want to tune the brightness you want to control the JPEG compression level and you 23:11.920 --> 23:17.200 can control the Alexa format etc there's also a video format for Deeper just like in Deeper 23:17.200 --> 23:25.120 to Deeper API are really inspired from Linux API. There is to get some of the same environments 23:25.120 --> 23:33.520 for people who are used to one or one of them and the drivers they also communicate directly 23:33.520 --> 23:38.000 for the controls that you don't have so much manual work to do for every detail so let's 23:38.000 --> 23:42.960 start to be a little bit more automation for all the drivers together so that it's not so bad or low 23:42.960 --> 23:48.480 level to bear it with here is how you permit to you can point the driver at each other that's the 23:48.480 --> 23:55.120 device to it that's a configuration language at Linux and Deeper used for configuring the device 23:55.680 --> 24:02.000 you declare a port you declare an endpoint and each endpoint is connected to the next device endpoint 24:02.160 --> 24:07.200 that's the way the drivers can know they can look up themselves what I am sending that are 24:07.200 --> 24:12.240 too and where do I get that from and then you can connect them to configure them with fewer manual 24:12.240 --> 24:20.640 walks this way that's the full chain from for such you can review that later with M2N with 24:21.280 --> 24:30.560 for the example area so we keep in mind that we still try to improve these APIs with 24:30.560 --> 24:35.760 more comfortable and easier to integrate with actual live software libraries that do 24:35.760 --> 24:44.080 image processing it's continuous evolution of these APIs so now let's look at what if you 24:44.080 --> 24:49.360 are example of what we can implement a video system like real real use case for for these APIs 24:49.360 --> 24:56.400 on how they are actually using in devices over time the domain of photography is giving birth to 24:56.400 --> 25:02.400 imaging and on top this which is a domain where you do not just want to watch the photo 25:02.400 --> 25:08.080 but extract information usually a human extract information from it will only also having in 25:08.080 --> 25:13.920 the last decade the last few years the rise of a computer vision when instead of human 25:13.920 --> 25:18.800 extracting information you don't need a computer that integrates a video function into this 25:18.800 --> 25:24.000 external ecosystem for example robotics or a lot of other use case which will be over game 25:24.960 --> 25:30.720 if we start from the route the phenomenon for the premise to sense lies from the computer system 25:30.720 --> 25:36.400 is photo diode it's permits to generate a very small voltage out of the other flight 25:37.040 --> 25:43.200 and if we just use a single photo diode if we put it on a on a mast on a tube and we rotate the 25:43.200 --> 25:50.560 tube and we can scan position one by one we get an image and that a single pixel in 25:50.560 --> 25:58.800 end one at a time scroll for it a bit and we get the full frame like that of course that 25:58.800 --> 26:03.920 it's possible to implement it with deeper that's API for it but usually you will want to 26:03.920 --> 26:09.920 get into the next general you also realize that work is just one footing so there's a 26:09.920 --> 26:17.120 CO2 sensors like the actual CO2 measurements of the Earth is done with the 26:17.440 --> 26:24.960 single pixel like this which is such kind of infrastructure so it's used in instrument 26:24.960 --> 26:29.280 it's scientific industry not that also genetic one thing which is done with photonics with 26:29.280 --> 26:33.760 emission source where it's only have a number to produce low emissions for for genetic 26:33.760 --> 26:38.880 concern and we have chemical process and as the end you will study all put into the chemical 26:38.960 --> 26:46.320 process with an image sensor on the anything control actually primitive is that but you will 26:46.320 --> 26:51.520 require to have a sensing the voltage out of the image sensor that's very kind of different 26:52.400 --> 26:58.400 from actual camera references so that's not what he's a line sensor a single line of pixel we have 26:58.400 --> 27:06.160 one pixel and we have a line of pixel that you will use that with ADC's and you will have to 27:06.160 --> 27:10.880 sense a voltage very fast one pixel at a time you will need kind of a specialize system 27:10.880 --> 27:17.680 on top of that and for instance that analog device will provide ADC for for deeper that you can 27:17.680 --> 27:24.240 use that you can actually implement a line sensor for instance for doing a scientific instrument 27:24.240 --> 27:31.040 using the deeper all that process using the tiny plants that we are building you can implement 27:31.040 --> 27:37.920 most of this logic using a deeper to right right today so now with multiple lines you actually 27:37.920 --> 27:44.240 get an image sensor the difficulty of embedded very small devices is to get access to a camera 27:44.240 --> 27:50.240 port and camera port I kind of not present on every device you can use adaptor chips so you can 27:50.240 --> 27:55.680 if you need imaging on a very very small device we look for my Raspberry Pi 27:55.680 --> 28:02.240 conference as you can always get video out of it so let's quickly look at what kind of 28:02.240 --> 28:07.680 processing you require to get an actual image out of real to the image sensor because what 28:07.680 --> 28:13.120 you get is that that the image you get out of image sensor or low resolution is normal but the 28:13.120 --> 28:19.520 color is completely weird you have to make several step of processing before you can actually 28:19.520 --> 28:26.480 get an image and you are more familiar with this kind of image for with normal looking color 28:26.480 --> 28:31.520 good contrast and feature so it means that every camera device has to do some processing in order 28:31.520 --> 28:36.480 to not look like that as well as that's really what you get out of the works that's one thing 28:36.480 --> 28:42.080 to implement so what why will you want to do all this step if you just do robotics you might 28:42.160 --> 28:49.600 does not have to look good it's okay if the computer looks at something green but it can have 28:49.600 --> 28:56.080 a few artifacts that are going to trick the robotic imaging system so you also have to do some 28:56.080 --> 29:02.320 kind of image processing for robotics so that you have a clean image without without artifacts 29:03.280 --> 29:11.200 so now let's look at actual different devices for making a better photo material you will have 29:12.240 --> 29:17.920 that's a sense of wavelength of light so that you can see the color spectrum of light 29:17.920 --> 29:24.560 and you will have to use a light like the one we saw earlier and so you need the very fast 29:24.560 --> 29:31.120 ADC and you really have to to as fast as possible wall along with it which we can 29:31.200 --> 29:35.760 implement that with deferral already that's the kind of image you get out and that's the kind of 29:35.760 --> 29:41.040 that you extract from the images to get the wavelength according to the light and for every compound 29:41.040 --> 29:47.600 chemical compound it's going to be different spectrum like that so if you want to implement a drone 29:47.600 --> 29:52.800 that's drone system available in deferred that's a robotics and motor control could call 29:52.800 --> 29:58.960 Kali pilots for auto pilot drone and it's deployed an independent on top of deferred and the wall 29:58.960 --> 30:03.760 that's the deferret so it's actually there on deferral as well as the kind of thing you could 30:03.760 --> 30:09.120 want to do for imaging along with a drone for instance that's Kali imaging with infrared 30:09.120 --> 30:15.120 that was a color not looks not green but fake color here and you can use open software to extract 30:15.120 --> 30:20.080 the images that's kind of use case you want to use very small embedded imaging imaging for 30:21.360 --> 30:25.120 you will for instance you want to use monitoring with taking a photo of your 30:25.120 --> 30:29.680 process you do not need a large resolution photo but you need to integrate a lot of all the sense 30:29.680 --> 30:36.720 for like CO2 the blink and LED ah thanks up well I'm I'm very sorry I just planned too much slides 30:38.000 --> 30:45.200 so it's yeah do I have five minutes okay thank you questions okay so that's a lot of 30:45.200 --> 30:50.880 example let's jump quickly let's just let's just continue to be able to review it 30:50.880 --> 31:00.880 ah that's a yeah too that's I just planned too many examples that's easy to cut all the content is there 31:00.880 --> 31:05.040 it's just to give you the idea of what it would represent to build videos devices with 31:05.040 --> 31:12.640 easy for today that's another example of how you could build a Bluetooth based my classes which 31:12.640 --> 31:17.680 you can really want to show you the video device physically and also for power with only one 31:17.760 --> 31:25.200 emissions of the chip and now I do not have to look at what's next coming in different videos 31:25.200 --> 31:30.240 sorry but this is coming next that means you will also be having news of it coming 31:30.240 --> 31:36.960 coming later but maybe something even more interesting is to ask to you what are your expectation 31:36.960 --> 31:42.400 regarding embedded video devices where you are after building small compact video devices where 31:42.480 --> 31:48.400 you're facing situation are you wanted to have something small as an Linux and did you ever try 31:48.400 --> 31:53.600 see for a PI for a video and do you have back for that or if you also have any sort of questions 31:53.600 --> 31:57.520 I will be very happy to hear about that 31:57.520 --> 32:14.720 so if you do not have questions we can never ask to look at the rest of the slides but 32:15.440 --> 32:19.520 yeah 32:22.000 --> 32:30.880 yeah 32:30.880 --> 32:34.880 these these I and exposure in the environment. 32:34.880 --> 32:35.880 Yes, of course. 32:37.880 --> 32:40.880 So do you mean exposure is up? 32:40.880 --> 32:41.880 Yes. 32:41.880 --> 32:49.880 Can you are asking if what happens about all the emissions of controls? 32:49.880 --> 32:51.880 Is that the question? 32:51.880 --> 32:52.880 Yes, no. 32:52.880 --> 32:56.880 We said that the remote sensor would have to adapt. 32:56.880 --> 32:57.880 Oh, here. 32:57.880 --> 32:58.880 Yes. 32:58.880 --> 33:01.880 So from this point of view, the data will make it easier. 33:03.880 --> 33:04.880 All right. 33:05.880 --> 33:06.880 Yes, absolutely. 33:06.880 --> 33:07.880 Good point. 33:07.880 --> 33:12.880 If you have image coming out like this, the first step. 33:12.880 --> 33:17.880 So you are asking, when you have some control on the image sensor itself, 33:17.880 --> 33:20.880 so that you can tune the image. 33:20.880 --> 33:21.880 For instance, auto exposure. 33:21.880 --> 33:22.880 That's a very good point. 33:22.880 --> 33:28.880 And auto exposure is one of the only feature that is only integrated in the image sensor on 33:28.880 --> 33:29.880 not processing on the chip. 33:29.880 --> 33:33.880 So the first two pictures are exactly the same. 33:33.880 --> 33:35.880 One with auto exposure on one without. 33:35.880 --> 33:40.880 It just has a default exposure is correct, but you have to constantly tune the image sensor. 33:40.880 --> 33:43.880 And that will be this kind. 33:45.880 --> 33:46.880 This cannot API. 33:46.880 --> 33:47.880 The control API. 33:47.880 --> 33:48.880 We have the software. 33:48.880 --> 33:50.880 Send command to the image sensor to adjust. 33:50.880 --> 33:51.880 I want more brightness. 33:51.880 --> 33:53.880 And you have to study the image. 33:53.880 --> 33:54.880 It is totally dark. 33:54.880 --> 33:56.880 Increase exposure. 33:56.880 --> 33:57.880 Or as the positive. 33:57.880 --> 33:58.880 It's way too bright. 33:58.880 --> 33:59.880 And you have to read this exposure. 33:59.880 --> 34:01.880 You can do the control with software. 34:01.880 --> 34:04.880 And you can send ice per se command to the image sensor. 34:04.880 --> 34:09.880 And this is why you need a lot of drivers so that you can know which command to send to the image sensor, 34:09.880 --> 34:10.880 which ice class here. 34:13.880 --> 34:15.880 There are other questions. 34:15.880 --> 34:17.880 Is that data sensor? 34:17.880 --> 34:18.880 Yeah. 34:21.880 --> 34:24.880 Oh. 34:24.880 --> 34:26.880 Oh. 34:28.880 --> 34:29.880 What did I get? 34:29.880 --> 34:30.880 That makes it.