· "A feature of the light field camera is that tens of millions of compound eye lenses of different angles are packaged on the surface of its COMS chip, each lens is the size of a hair, equivalent to the function of a camera to achieve multiple cameras, and capture and record information of different dimensions of light in space as much as possible."

"If you use a light-field camera to capture real-world information, then use computer modeling and input it into VR glasses, you can build the entire three-dimensional world of the meta-universe more realistic."

Bionic insect compound eye imaging, hundreds of thousands to millions of tiny compound eye sensors precision packaging into a camera, a single shot to achieve three-dimensional imaging... This is no longer the cutting-edge research in the top issue of the paper, light field cameras have moved out of the lab and have a real role in industrial Settings.

"A feature of the light field camera is that the surface of its COMS chip is packaged with tens of millions of compound eye lenses of different angles, each lens is the size of a hair, equivalent to a camera to achieve the function of multiple cameras, as far as possible to capture and record the different dimensions of light in space." Yi mu technology founder Li Haotian in an interview with surging news introduced.

With the continuous development of intelligent manufacturing, machine vision, as the eye of the production line, directly restricts the factory quality and safety of products. Although China has the world's largest manufacturing market, most of the core vision modules are not domestically produced. Yimu Technology, founded in 2019, is the first 3D light field camera provider in China to fully grasp the full link core technologies of optical design, micro and nano processing, packaging and manufacturing, algorithm software, etc. In 2021, Yimu Technology took the lead in realizing the batch application of light field cameras in industrial production line detection.

In the interview, Li Haotian explained the principles of the emerging light field imaging technology, introduced the application scenarios of light field technology, and its role in the production of core parts of the metauniverse, and also talked about the technical trend and planning.

The following is part of the interview transcript:

Thepaper.cn: What is light field? What is the principle of light field camera?

Li Haotian: In fact, the light field is similar to the magnetic field and gravity field learned in physics, which can not be seen or touched, but exists in all aspects of life. Why is it called light field? Because light has many dimensions, different wavelengths represent different colors, such as long wavelengths of red light, short wavelengths of violet light; There is also polarization, different polarization states can present different display states; And the intensity of light, like light and dark; Light travels in straight lines and so on. Light field is a unified concept that describes all the properties of light.

A feature of the light field camera is that tens of millions of compound eye lenses of different angles are packaged on the surface of its COMS chip, each lens is the size of a hair, equivalent to a camera to achieve the function of multiple cameras, and capture and record the information of light in space as much as possible.

Thepaper.cn: What are the current applications of light field cameras?

Li Haotian: In fact, at present in foreign countries, the light field camera is most talked about in the field of meta-universe, which can complete the information collection at the front of the meta-universe.

The most critical core components of the meta-universe are the VR glasses and the naked eye 3D. In the real scene, the human eye can constantly focus near and far, such as focusing in the distance, near is virtual focus. But generally VR glasses are fixed focal length, only two-dimensional information, then the eyes will be tired, but also feel that the information is not real. Today's VR glasses can indeed show 3D characteristics of different perspectives, but if the Angle is relatively large, or there are many people on the same screen, it may not be accurate enough. Because at the source, it can't capture the light from which direction the eye sees it. If you use the light field camera to capture the real world information, and then use the computer modeling and input into the VR glasses, you can build the whole three-dimensional world more realistic.

Thepaper.cn: You mentioned that such devices as VR are currently immature, so what will the final form be like after maturity?

Li Haotian: I think the final form will be like the science fiction movies we watch. Imagine a scenario where you wake up every morning and wear a special suit that has a bunch of sensors on it, but we can't see or touch it and feel comfortable. For example, the sensor on the finger, the hand can reach to touch some virtual things; The sensor in my head can sense my mood today, and if I am not happy, the meta-universe may push something happy; At the same time, some virtual images are projected on the eyeball. After washing up and going to work, you can see a variety of virtual three-dimensional naked eye 3D equipment throughout the city. It's like a scene from a science fiction movie, but in some ways one of the most important drives of technological development is the realization of human imagination and needs.

Thepaper.cn: According to the information you have learned, what is the current development rate of China in the metaverse related industries?

Li Haotian: At present, at the hardware level, such as VR glasses hardware, China is in line with the world's most first-class manufacturers, the latest international technology, the latest production needs are in China, China can definitely catch up in the hardware level. China has Pico, Bytedance, including Xiaomi, Huawei have also launched a series of VR glasses, I believe in the brand, in the software system has a certain breakthrough. Therefore, looking at the whole, China is certainly not far from the world's top level in the field of meta-space.

Thepaper.cn: Can you explain in a popular way what are the core components of VR, AR, and XR devices, and what role will light field technology play in them?

Li Haotian: We can simply understand VR as a display module, just like a mobile phone, it has a display inside, but its display may be very small. Then it has optical imaging lens modules, some VR glasses are relatively thin, some are relatively thick, and the specific optical path design of its imaging mode is completely different.

In addition to the acquisition end of the meta-universe mentioned earlier, the light field is also helping to produce more stable and better quality VR equipment at this stage. For example, the light field camera has now been involved in some production lines of VR glasses to help them detect the defects of VR glasses, including the defects of the virtual phase of its imaging and the distance problem.

The left and right eyes of the VR glasses are independently imaged, and if the left eye casts like 1 meter and the right eye is 1.5 meters, you may get dizzy when wearing the glasses. For us, we can detect the distance between the transparent virtual phase surface and the real eye, with up to a millimeter accuracy in the range of 1.5 meters.

For a lens module, it has several layers of lenses, each layer may have a coating, polarizer, etc. There may be defects, bubbles or spots in the film, and before it is assembled into VR glasses, the light field camera can locate the defects at which layer, and some defects can be repaired.

Thepaper.cn: What are the special advantages of light field cameras to do this detection?

Li Haotian: The light field can obtain information from different angles in one shot, which is equivalent to multiple cameras shooting at the same time, as well as information from different Angle planes. And this information is passive imaging, unlike other 2D, 3D equipment such as structured light, line laser equipment, such as liDAR, it has to emit laser out, reflected back to detect. But such technology once encountered like transparent glass, or virtual phase in VR glasses, are not detectable. Passive imaging of light field can solve these problems well.

The Paper: Light field is a very new technology, you once mentioned in an interview that a major task in the next few years is to reduce costs, what is the current level?

Li Haotian: Now we are mainly focused on the high-end intelligent manufacturing industry in the industrial field, so the price of a single unit is still relatively high. We are now also working to reduce prices and expand to more markets, by optimizing the supply chain, to develop more low-cost light field cameras.

Thepaper.cn: What does the cost reduction depend on?

Li Haotian: The first is the quantity, and the second is the maturity of the technology. Now the technology is still iterating, at this stage, we will not buy some parts in large quantities such as the core chip inside. Like a mobile phone camera, it is tens of millions of pixels at the beginning, and now there are several hundred million pixels in the back, in fact, it is to reduce the cost by increasing the amount.

For this industrial sensor chip, it can also reduce the cost through volume, and it is necessary to iterate step by step.

Thepaper.cn: I know you have a technical background. What was the process of finding application scenarios for such an emerging technology as light field at the beginning? Is the future planning to go deep into the upstream and downstream of some industries, or to horizontally expand different application scenarios with technology as the core?

Li Haotian: In the beginning, we actually kind of looked for nails with a hammer, but we have been trying to use this hammer well. When I came out to start my business, I thought that since it can measure a few microns in the laboratory, including such sophisticated things as aircraft engines, it must also be able to do some three-dimensional modeling. Around 2015, there was a craze for VR glasses, and we thought that light fields could be reconstructed in 3D.

Later, after officially coming out to start a business in 2019, the entire market was not particularly good, and VR glasses did not have any news, so we went to find some new possibilities. When a Korean company came to us and said that light field cameras might solve the problem of screen defect detection, we flew a dozen times to Korea and achieved the first set of sales.

For the future planning, because our team is a technical team, we are better at doing the iteration of the technology itself, focusing on the imaging of the light field itself, and empowering this technology in all walks of life, which is what we are more willing to do and have the ability to do.

Attachment: The technical development process of Yimu technology

In 2015, the first set of scientific research grade light field 3D imaging system was developed;

In 2016, the micrometer-level three-dimensional measurement accuracy of particles was realized;

In 2018, we independently developed optical field acquisition chip packaging and testing equipment;

Localization of all core components in 2019;

In 2020, we successfully developed an industrial-grade microscopic light field camera.

In 2021, the first batch application of light field cameras in industrial production line detection was realized.