The hybrid of the camera and lidar improves the capabilities of the car, supplementing the information on the external world
Lidars and cameras are two standard configuration elements for almost any robotic vehicle. Both the first and second work with reflected light. Cameras in this case work in passive mode, that is, they catch the reflection of external sources of illumination, but lidars generate laser pulses, then measuring the "response" reflected from nearby objects. Cameras form a two-dimensional image, and lidars - three-dimensional, something like a "cloud of points."
Company Ouster has developed a hybrid device , which works as a camera, and as a lidar. This system is called OS-1. This device has a larger aperture than most DSLRs, and the sensor created by the company is very sensitive.
The images received by the system consist of three layers. The first is an image obtained as if by a normal camera. The second is a "laser" layer obtained using laser beam reflection. And the third is a "deep" layer, which allows you to estimate the distance between the individual pixels of the first two layers.
It is worth noting that the images still have significant limitations. First, these are low-resolution images. Secondly, they are black and white, not colored. Thirdly, the lidar does not work with a visible light source, it deals with a spectrum close to infrared.
At the moment, the cost of the lidar is quite high - about $ 1?000. At first glance, the sense in the system, which receives images of lower resolution than the standard cameras, but costs, like a cast-iron bridge, no. But the developers argue that there is a different principle of work here than in the usual case.
These are graphic materials provided by Ouster. Here are three layers of images and a common "picture", which is the result of
In a typical situation, the car drives combine data from several different sources, which takes time. Cameras and lidars work in different modes, the result of the work is also different. In addition, they are usually mounted in different places of the car's body, so the computer has to also correlate images to make them compatible. Moreover, the sensors require regular recalibration, which is not so easy.
Some lidar developers have already tried to combine the camera with the lidar. But the results were not very good. It was a "standard camera + lidar" system, which was not too different from the existing ones.
Ouster instead uses a system that allows OS-1 to collect all data in one standard and from one location. All three layers of the image perfectly correlate, both in time and in space. In this case, the computer understands the distance between the individual pixels of the final image.
According to the authors of the project, such a scheme is almost ideal for machine learning. For computer systems, processing such images is not difficult. "Feeding" the system several hundred shots, it can be trained to accurately understand what is depicted in the final "picture".
Some types of neural networks are designed in such a way that without problems to work with multilayers of pixel maps. In addition, images can contain red, blue and green layers. Teach such systems to work with the result of OS-1 work is not particularly difficult. The company Ouster has already solved this problem.
As a source material, they took several neural networks that are designed to recognize RGB images, and modified them to suit their needs, by teaching them to work with different layers of their images. Data processing is carried out on equipment with Nvidia GTX 1060. With the help of neural networks, the computer of the car was taught to "paint" the road in yellow, and potential obstacles - other cars - to red.
According to the developers, their system is a complement to the already existing, and not a replacement. It is best to combine all sorts of sensors, sensors, cameras, lidars and hybrid systems to form a clear picture of the environment, which will help the car to navigate.
It may be interesting
visit this site
visit this site
visit this site
Pleasant data, significant and magnificent plan, as offer great stuff with smart thoughts and ideas, bunches of incredible data and motivation, the two of which I need, on account of offer such an accommodating data here.