Interview Background: In the field of industrial manufacturing, machine vision is mainly for semiconductor and electronic manufacturing, automobile manufacturing, food and packaging, biopharmaceutical and other industries, to achieve defect detection, size measurement, pattern recognition, navigation and positioning functions, so as to greatly improve product quality and production efficiency, and ensure the safety of industrial site environment. The emergence of 3D machine vision not only effectively solves the problems of complex object recognition and depth measurement in the traditional 2D era, but also realizes more complex human-computer interaction functions, and becomes an important promoter of industrial automation and intelligence.

At present, 3D machine vision technology mainly includes binocular vision, structured light and laser triangulation. Today, we are going to introduce a new 3D vision technology - light field imaging. Established in 2019, YIMu (Shanghai) Technology Co., LTD. (hereinafter referred to as "YIMu Technology") has become one of the few optical field 3D imaging technology pioneers in the world who have fully mastered all the core technologies of optical field acquisition chip optical design, manufacturing, sealing, three-dimensional rendering of optical field image and three-dimensional precision and rapid detection. The accuracy and efficiency of optical field 3D measurement is at the international leading level. On September 16 this year, Yimu technology will also be in『The 31st "Small Words" Symposium: 3D Vision Technology and applications』In order to understand 3D light field camera technology and applications in advance, Mames consulting specifically interviewed Li Haotian, general manager and co-founder of Yimu Technology.

Yi Mu technology general manager, co-founder Li Haotian

Myms Consulting: Thank you for being interviewed by Myms Consulting. Yimu Technology is a very young company, but it has already achieved remarkable results. Please introduce the development process of the company to everyone, thank you!

Li Haotian: Yimu Technology was founded in July 2019. Although I am very young, I have already actively carried out the early technical research and accumulation during my study in Shanghai Jiao Tong University. With the full assistance of the senior scientific research team of Shanghai Jiao Tong University, I have applied the new three-dimensional imaging technology of light field camera to industrial inspection step by step. In 2015, the first set of scientific research grade optical field 3D imaging system was successfully developed, the next year optical field imaging technology achieved micron-level measurement accuracy in the field of three-dimensional flow testing, the acquisition chip packaging and test equipment was successfully developed independently in 2018, all core components were localized in 2019, and the industrial-grade microscopic light field camera was successfully developed in 2020, and was reported by CCTV. In 2021, the optical field camera will be applied to the production line detection of MEMS silicon chips on a large scale, which is also the first time in the world.

Yimu Technology was born in the "Zero Bay" global innovation and entrepreneurship cluster, backed by Shanghai Jiao Tong University, and deep cooperation with Singapore Nanyang Technological University, quickly realize technology update and product iteration; At the same time, "Zero Bay" also provides a steady stream of "nutrients" to the team in many aspects such as "policy support, financing services, and legal advice". Young Yimu Technology has become the world's second, the exclusive domestic comprehensive master of industrial light field camera in optical design, simulation, micro and nano processing, packaging testing, three-dimensional precision rapid detection and three-dimensional rendering of light field image and other fields of all the core technologies of high-tech companies.

Through deep cooperation with upstream and downstream leading enterprises Changguang Chen Core (Gpixel) and Huaxing Yuan Chuang (688001.SH), Yimu Technology's light field three-dimensional detection system has been successfully applied to a variety of industrial detection fields: Three-dimensional detection of semiconductor bond alloy line, three-dimensional layered detection of screen defects, three-dimensional layered detection of polarizer module, three-dimensional layered detection of grating module, three-dimensional layered detection of new energy vehicle battery module.

Yimu technology growth track

Myms Consulting: When it comes to 3D vision technology, the most familiar technologies should be binocular vision, structured light and laser triangulation (line laser). And for the light field imaging method, you may not know enough. Can you do some science popularization for us first?

Li Haotian: Many insects, such as dragonflies, have a kind of visual organ that we call the compound eye. A single compound eye has tens of thousands of independently sensitive eyes, which are densely arranged in hexagons, providing insects with a good field of view and strong three-dimensional perception. Inspired by the compound eye of insects, scientists have proposed an imaging method that simulates the compound eye of insects using an array of many tiny lenses. This unique imaging method can not only record the color information of the object, but also accurately capture the three-dimensional trajectory of the light, and through a single shot, the multidimensional light field information in space can be completely recorded, so it is called "light field camera". Further through the optoelectromechanical coupling 3D rendering algorithm of light field, the optical field camera can restore the captured 3D scene at a very fast speed, and realize real-time 3D reconstruction and 3D measurement.

The light field camera based on compound eye bionics launched by Yimu Technology

Mames Consulting: Yimu Technology's 3D light field vision technology is mainly for industrial inspection applications, compared to structured light, line laser, etc., what are the advantages of your technology?

Li Haotian: The existing mature three-dimensional detection schemes for machine vision are mainly structured light and line laser.

Structured light inspection system consists of projectors and ordinary industrial cameras, by projecting a specific coded pattern, using the collected surface texture of the object to calculate three-dimensional information. The line laser system consists of a laser transmitter and an ordinary industrial camera, and measures the three-dimensional size of an object according to the laser triangulation method. These two technologies have been very mature, the measurement accuracy and efficiency are high. Whether it is structured light projection module, laser transmitter or industrial camera, there are already a variety of mature products to choose from, and its core algorithm has been open source, reducing the entry threshold of structured light and line laser detection system, and promoting the application of these two technologies in the field of industrial detection.

Whether it is structured light or linear laser, it is necessary to actively project a specific light source, and the structured light projector and camera or the laser transmitter and camera need to be arranged strictly according to a certain Angle, which is easy to produce occlusion phenomenon when measuring complex objects; At the same time, it is difficult for active projective structural light source or line laser to image on transparent medium and detect internal defects of transparent medium. In contrast, the light field camera has a unique advantage for the three-dimensional measurement/detection of transparent and highly reflective materials, only the ambient light source, a single camera, a single shot, can complete the three-dimensional measurement/detection, and there is no problem of occlusion.

The difference between light field camera and structured light camera and line laser camera

Myms Consulting: What core technologies has Yimu Technology mastered for 3D light field cameras? Among them, which core technology has the highest industry barriers? In addition, what kind of layout do you have in terms of intellectual property rights?

Li Haotian: The optical field camera covers a variety of key technologies such as micro and nano optical design, simulation, manufacturing, detection, chip-level packaging, testing, fast and high-precision computational imaging, and each single core technology has quite high industry barriers. How to organically integrate optical design, micro and nano manufacturing, sealed measurement and light field algorithm into a whole to achieve the demanding demand for high-speed and high-precision detection of industrial production lines is the most difficult challenge. Yimu Technology has mastered the core technology of the whole chain from optical field camera design, micro and nano manufacturing, packaging and testing, three-dimensional precision and rapid detection algorithm of optical field to large-scale application of production line, and achieved stable industrial applications of millisecond measurement speed and sub-micron measurement accuracy.

Yimu Technology attaches great importance to the protection of core technologies and intellectual property rights, and continuously strengthens the management of intellectual property rights. All technical documents within the company are confidential, and there is a strict approval and decryption management process. At the same time, the company timely research and development of the hardware technology, core algorithms and software to apply for patents and Copyrights. Through the combination of technical confidentiality and intellectual property application, the company's core technology is protected. Up to now, Yimu Technology is applying for and has authorized more than 50 PCT patents, invention patents and software Copyrights.

Myms Consulting: The algorithm also determines the final image quality of the 3D light field camera. What are the "unique secrets" of Yimu technology in this respect?

Li Haotian: The light field algorithm of Yimu Technology is rooted in the scientific research achievements of Shanghai Jiaotong University for many years. At the same time, it constantly absorbs the application experience of production line. Through repeated grinding and optimization iteration of several industrial testing projects, it has formed an efficient, high-precision, stable and reliable three-dimensional light field algorithm suitable for industrial production line detection. Based on a large number of production line data and the integration of deep learning, the artificial intelligence three-dimensional reconstruction algorithm of Yimu Technology has ranked first in the global authoritative ranking of light field algorithms, ranking first in the world in terms of accuracy and computing efficiency. Not only that, Yimu technology research and development personnel are immersed in the front end of the production line, listen to customer feedback, and actively develop a universal light field industrial testing platform, which greatly improves user interaction performance and product adaptability.

MEMS Consulting: Yimu Technology's 3D light field camera has been used in MEMS microphones lead key alloy wire defect detection, can you introduce the application case? The semiconductor industry has relatively high requirements on the accuracy of inspection equipment. What is the accuracy of your camera at present?

Li Haotian: Due to the small size, compact structure, complex spatial structure of MEMS microphone chip, small diameter of the chip upper key alloy wire, complex shape, and easy reflection of the material, the existing three-dimensional detection equipment can not meet customer needs. Because of its passive imaging, the light field camera can capture images from multiple perspectives at the same time, avoiding the interference caused by the complex structure and specific Angle reflection. It only takes 50ms to 100ms to obtain 3D information, so it is particularly suitable for the high precision and fast detection needs of MEMS. Another significant advantage of light-field cameras is that different precision can be achieved by matching different optical lenses. At present, the highest depth direction accuracy of the industrial light field camera of Yimu Technology is 2 microns, and the repetition accuracy is 0.5 microns.

According to our understanding, Yimu Technology has cut into the international 3C giant industry chain, used for wireless Bluetooth headsets, mobile phone cameras and mobile phone screens layered detection. Can you talk about the progress so far?

Li Haotian: Our optical field detection system has been applied to the TWS chip production line of an international 3C head enterprise, which solves the problem that the existing 2D detection platform cannot detect the three-dimensional defects of the chip online. Our optical field industrial inspection platform software can be adapted to multiple inspection projects of dozens of chips, and currently detects more than 10,000 UPH (production capacity per hour), with a depth of field accuracy of up to 6 microns and a plane accuracy of up to 1 micron. The system will also soon be applied to another international 3C head enterprise chip production line.

The three-dimensional light field layered detection system for display module defects has also been running on the production line of a head enterprise, and the visual scheme can accurately locate the three-dimensional position of various defects such as adhesion dirt, adhesion foreign matter, and bubbles of the display module. At the same time, the system can also be applied to polarizer module detection, help customers locate defects, improve delivery yield, trace the link of defects, optimize the production process, reduce labor costs.

The camera module 3D defect detection solution can be used to inspect not only the complete module, but also single-layer lenses. Identify and locate defects such as dirt, bubbles, white spots, line damage, and stencil in the lens module and defects such as damage, bubbles, and dirt on the single-layer lens coating.

In addition, the optical field detection system of Yimu Technology has also obtained good production line testing results in the fields of CIS (CMOS image sensor) protective glass defect detection, mobile phone CGS (continuous granular silicon) module detection, and three-dimensional defect detection of new energy vehicle batteries.

Yimu Technology 3D light field camera semiconductor, display module defect detection example

Mames Consulting: Which domestic and foreign enterprises do your 3D light field cameras target? What are your strengths?

Li Haotian: Looking at the world, there are also German Raytrix companies that apply light field technology to the industrial field. Compared to the Raytrix optical field camera, the EMo optical field camera uses a completely different optical field imaging principle, which is essentially more accurate in the depth of field direction. In addition, Yimu Technology has mastered all the "soft and hard" core technologies of light field cameras, and all the core components have been localization. Therefore, it has significant advantages in terms of overall system price, delivery cycle, rapid technical support and customized services.

Yimu Technology industrial grade microscopic light field imaging system

Maims Consulting: The COVID-19 epidemic has accelerated the pace of industrial automation and promoted the application of vision sensors. Please talk about the progress of product sales of Yimu Technology.

At the same time, we will further expand the application market, increase sales revenue through direct sales and agent sales, and strive to become the international 3D vision program leader.

Mames Consulting: Looking ahead, what are the other exciting application directions for 3D light field cameras?

Li Haotian: The light field camera can be applied to many directions in the future because of its unique light field characteristics. These include:

（1）Combined with the automated inspection platform, it is applied to three-dimensional visual inspection projects in conventional industrial fields, such as MEMS chips, display devices, camera modules, new energy battery modules, etc.;

（2）Light field cameras with high-power microscopic objectives can be used in the biomedical field, especially in the three-dimensional detection of cells in vivo;

（3）The light field camera can also be applied to the field of virtual reality, such as providing three-dimensional image input for the light field true 3D display, AR, VR;

（4）As a security, vehicle three-dimensional perception module.

Mames Consulting: Please talk about the development plan of Yimu Technology in the next three years, thank you!

Li Haotian: The development of Yimu Technology in the next three years is mainly focused on the field of industrial 3D machine vision. The goal of 2021 is to realize the batch application of optical field technology in the production lines of leading enterprises in many fields; In 2022, the micro-nano machining center was successfully established and put into operation, and a number of sales and technical support centers were established in East and South China, and the large-scale application of optical field technology in various industrial scenarios was realized. 2023 gradually expand the application of light field technology in biomedicine, unmanned driving, security, vehicle, VR and other fields.

Myms Consulting: Do you have any plans to raise capital in the near future? If so, can you clarify the need and use of the funds?

Li Haotian: The company will launch A round of financing in the near future, this round of financing will be tens of millions of level, the funds are mainly used for the research and development of key technologies, market laying, production capacity improvement, and team building. Up to now, we are actively communicating with a number of domestic first-line investment institutions, and the intended investors are optimistic about the broad market dividends of the machine vision industry in the future.

Mames Consulting: Welcome to the event held by Yimu Technology on September 16-17, 2021『The 31st "Small Words" Symposium: 3D Vision Technology and applications』Can you give us a preview of what you want to share with the audience?

Li Haotian: Light field camera as a new three-dimensional imaging technology is not well known to the public. With the maturity of technology and the iterative upgrading of products, optical field imaging technology has been able to solve some difficult points in many industrial scenes. This lecture will focus on the principle of light field imaging, the introduction of the characteristics of the light field camera of Yimo Technology, and the sharing of successful application scenarios, especially the application of chip gold wire detection and screen defect layered detection in the two fields, hoping to help the majority of industrial machine vision practitioners and related demand parties to broaden their horizons and increase their cognition of light field.