News
iCatch’s SoCs are powering a diversity of camera applications in
multiple markets.
Blog
Mar 31, 2021
Image Signal Processor (ISP) Introduction
The ISP is a kind of multimedia processor or an application specific digitial signal processor, typically in a chip form, which can generate a human perceivable image from a digital image signal. Please refer to the diagram shown below, the image sensor catches the light passing through a lens and converts the light into digital signals of a raw image. The image signal processor (ISP) receives the raw image siangls to rebuild the full color image of primary components (red, green and blue color) which is suitable for human viewing. If the ISP chip can process the images in a rate of or more than 30 frames per second, it can be used for video applications. iCatch Technology is dedicating its R&D efforts in ISP design and forcuing the applications on automotive, home security and surveillance, and consumer electronics market to enable a safer and more convenient living of human life. The following paragraphs will introduce the key features of an ISP engine in more details.
As you can see the cameras are already widely used in new cars sold recently, like dash camera, electronic mirror, rear view camera, and around view camera. All of these cameras reqruied an advanced ISP to process the image signal. The processed image will be transferred to another processor for display, recording, surround view integration, switching bewteen camera views, and so on.
In morden image applications, the captured image is not only for human viewing but also for AI processing. As the noise in the image may interfere the AI prediction and increase the false judgment rate and time. A much clearer image quality can help AI to more accurately and efficiently perform artificial intelligence applications such as pedestrian detection on the road, personal access control, license plate recongnition, animal recognition for ecological conservation, factory robot automation, etc.
*High Dynamic Range or HDR is a technology to resolve the problem in the high brightness contrast scene which suffers from too dark in the dark and too bright in the bright spots. See the pictures below as example, the HDR function can simultaneously reduce the brightness of the extremely bright area (the sun and sky) and enhance the image detail in the dark areas.
According to the Wikipedia description [2], the illuminance under the scorching sun can reach 100,000 (105) lux and the illuminance of full moon is 0.2(10-1)lux. So the range of illuminance between brightness and darkness spans across six orders of magnitude.
Linear Range is 105 / 10-1 = 106
Dynamic Range is 20 log 105 / 10-1 = 120dB
Since automotive image sensing needs to handle harsh high brightness constract scenes (tunnel and direct sun light) and to maintain good image quality for both day and night time, an automotive camera module needs to have at least 120dB of dynamic range capability[3]. The iCatch Technology’s automotive ISP series product, Ci3-P, supports up to 144dB dynamic range. This ISP chip can dynamically adjust the dynamic range based on the currently scene condition and generate optimal images for the drivers or autonomous driving systems as a basis for judgment.
*The recognizability of car license plates is very critical for dashcam, especially in harsh blacklight scenes. Check the picture below for example, the left image is made by iCatch Technology’s M3 ISP chip and the right image is made by referenced ISP. As you can see in the comparison, the M3 ISP can provide identifiable license plate numbers on the vehicle in the front under the backlight condition at day time. When M3 ISP is used in a rear camera, it can also provide identifiable car license plate numbers of the vehicle in the back with headlights on at night.
In the experiment of exploring the relationship between recognizability of license plate number and the maximum distance, the iCatch’s M3 ISP testing module consists a FHD resolution image sensor and a lens assembly with F/N (F Number) 1.7 and HFOV (Horizontal Field of View) 115°. The results reveal that the iCatch M3 ISP can provide a clear license plate image for recognition within two vehicle body distances. See the pictures below which are taken at distance of 6 meters and 9 meters from the vehicle, respectively. In the distince of 6 meters, the English letters and numbers can be clearly recognized. On the other hand, only parts of the English letters and numbers can be recognized in the distance of 9 meters.
The camera lens can influence recognizable distance of the license plate [4]. Under the same resolution condition of an image sensor, the smaller the FOV is, the more the information of license plate can be obtained. If the FOV of the selected lens is 1/2 smaller than the lens used in this testing module, theoretically the recognizable distance of the license plate can be doubled.
The image sensor resolution can also influence the recognizable distance of the license plate. The higher the sensor resolution is, the more detail image information the sensor can catch. If the resolution of selected sensor is twice that of the sensor used in this testing module, the recognizable distance of the license plate will also be doubled.
License plate recognition system that usd iCatch Technology’s M3 series ISP can clearly identify licesne plate number within the distance of two car bodies. Under the same resolution conditionof a 2Mpixel image sensor, a smaller FOV (Field of View) lens can recognize license plate in farther distince. On the other hand, a higher resolution sensor can also incress recognizable distince. The iCatch Technology’s Ci1 intelligent ISP can support 4Mpixel resolution sensor.
*Lens Distortion Correction (LDC) is a function for compensating the image distortion due to the lens module. Even high-end or expensive lenses will have some lens distortion problems. The main reason is the physical structure of lenses. See the below pictures as example, the image at the right is without correction and the image at the left is being corrected by the built-in LDC engine of the ISP. It is obvious that the image at the left is closer to the real image that human eye sees. [1]
iCatch Technology’s M3 series ISP include LDC engine. It can correct the distortion of the lens and restore a scene similar to what the human eys sees.
* HDR joint with 3D denoise technology, make the night image clearer.
The performance of HDR can greatly improve the image quality in the dark areas of an image. But at night time, there will still be noise particles in the image due to insufficient brightness. The iCatch Technology’s ISP solution has built-in a 3D denoise technology, which can reference privous frames and denoise the current image frame to generate a clear night sense. See the above image at left which is processed by iCatch’s ISP for example.
Conclusion
iCatch Technology’s ISP technology has a leading HDR performance and can provide clear images for drivers and for AI processors as a basis for judgment. The English letters and numbers of license plate can be clearly identified within the distance of two car bodies, and partially identified within the distance of three car bodies. The lens distortion correction function compensates the image distortion due to lens design. The 3D denoise technology can greatly reduce the image noise at night and improve the image quality. The iCatch’s Ci3 series ISP has also passed the AEC-Q100 Grade 2 certification, which can meet the high reliability requirement of automotive OEM companies on semiconductor chips and can be used in automotive pre-installation camera module. If you would like to know more details about iCatch Technology’s ISP product line, welcome to contact us.
For more information about iCatch’s products, please visit www.icatchtek.com
For sales contact, please leave your message at www.icatchtek.com/Requests
Reference
[1] Wikipedia, Perspective control https://en.wikipedia.org/wiki/Perspective_control
[2] Wikipedia, Illuminance https://en.wikipedia.org/wiki/Illuminance
[3] Sony, Key technology for addressing sensing application in automotive https://www.youtube.com/watch?v=1rkQFOxlMZg&feature=share
[4] EETimes, ADAS Front Camera: Demystifying Resolution and Frame-Rate
As you can see the cameras are already widely used in new cars sold recently, like dash camera, electronic mirror, rear view camera, and around view camera. All of these cameras reqruied an advanced ISP to process the image signal. The processed image will be transferred to another processor for display, recording, surround view integration, switching bewteen camera views, and so on.
In morden image applications, the captured image is not only for human viewing but also for AI processing. As the noise in the image may interfere the AI prediction and increase the false judgment rate and time. A much clearer image quality can help AI to more accurately and efficiently perform artificial intelligence applications such as pedestrian detection on the road, personal access control, license plate recongnition, animal recognition for ecological conservation, factory robot automation, etc.
*High Dynamic Range or HDR is a technology to resolve the problem in the high brightness contrast scene which suffers from too dark in the dark and too bright in the bright spots. See the pictures below as example, the HDR function can simultaneously reduce the brightness of the extremely bright area (the sun and sky) and enhance the image detail in the dark areas.
According to the Wikipedia description [2], the illuminance under the scorching sun can reach 100,000 (105) lux and the illuminance of full moon is 0.2(10-1)lux. So the range of illuminance between brightness and darkness spans across six orders of magnitude.
Linear Range is 105 / 10-1 = 106
Dynamic Range is 20 log 105 / 10-1 = 120dB
Since automotive image sensing needs to handle harsh high brightness constract scenes (tunnel and direct sun light) and to maintain good image quality for both day and night time, an automotive camera module needs to have at least 120dB of dynamic range capability[3]. The iCatch Technology’s automotive ISP series product, Ci3-P, supports up to 144dB dynamic range. This ISP chip can dynamically adjust the dynamic range based on the currently scene condition and generate optimal images for the drivers or autonomous driving systems as a basis for judgment.
*The recognizability of car license plates is very critical for dashcam, especially in harsh blacklight scenes. Check the picture below for example, the left image is made by iCatch Technology’s M3 ISP chip and the right image is made by referenced ISP. As you can see in the comparison, the M3 ISP can provide identifiable license plate numbers on the vehicle in the front under the backlight condition at day time. When M3 ISP is used in a rear camera, it can also provide identifiable car license plate numbers of the vehicle in the back with headlights on at night.
In the experiment of exploring the relationship between recognizability of license plate number and the maximum distance, the iCatch’s M3 ISP testing module consists a FHD resolution image sensor and a lens assembly with F/N (F Number) 1.7 and HFOV (Horizontal Field of View) 115°. The results reveal that the iCatch M3 ISP can provide a clear license plate image for recognition within two vehicle body distances. See the pictures below which are taken at distance of 6 meters and 9 meters from the vehicle, respectively. In the distince of 6 meters, the English letters and numbers can be clearly recognized. On the other hand, only parts of the English letters and numbers can be recognized in the distance of 9 meters.
The camera lens can influence recognizable distance of the license plate [4]. Under the same resolution condition of an image sensor, the smaller the FOV is, the more the information of license plate can be obtained. If the FOV of the selected lens is 1/2 smaller than the lens used in this testing module, theoretically the recognizable distance of the license plate can be doubled.
The image sensor resolution can also influence the recognizable distance of the license plate. The higher the sensor resolution is, the more detail image information the sensor can catch. If the resolution of selected sensor is twice that of the sensor used in this testing module, the recognizable distance of the license plate will also be doubled.
License plate recognition system that usd iCatch Technology’s M3 series ISP can clearly identify licesne plate number within the distance of two car bodies. Under the same resolution conditionof a 2Mpixel image sensor, a smaller FOV (Field of View) lens can recognize license plate in farther distince. On the other hand, a higher resolution sensor can also incress recognizable distince. The iCatch Technology’s Ci1 intelligent ISP can support 4Mpixel resolution sensor.
*Lens Distortion Correction (LDC) is a function for compensating the image distortion due to the lens module. Even high-end or expensive lenses will have some lens distortion problems. The main reason is the physical structure of lenses. See the below pictures as example, the image at the right is without correction and the image at the left is being corrected by the built-in LDC engine of the ISP. It is obvious that the image at the left is closer to the real image that human eye sees. [1]
iCatch Technology’s M3 series ISP include LDC engine. It can correct the distortion of the lens and restore a scene similar to what the human eys sees.
* HDR joint with 3D denoise technology, make the night image clearer.
The performance of HDR can greatly improve the image quality in the dark areas of an image. But at night time, there will still be noise particles in the image due to insufficient brightness. The iCatch Technology’s ISP solution has built-in a 3D denoise technology, which can reference privous frames and denoise the current image frame to generate a clear night sense. See the above image at left which is processed by iCatch’s ISP for example.
Conclusion
iCatch Technology’s ISP technology has a leading HDR performance and can provide clear images for drivers and for AI processors as a basis for judgment. The English letters and numbers of license plate can be clearly identified within the distance of two car bodies, and partially identified within the distance of three car bodies. The lens distortion correction function compensates the image distortion due to lens design. The 3D denoise technology can greatly reduce the image noise at night and improve the image quality. The iCatch’s Ci3 series ISP has also passed the AEC-Q100 Grade 2 certification, which can meet the high reliability requirement of automotive OEM companies on semiconductor chips and can be used in automotive pre-installation camera module. If you would like to know more details about iCatch Technology’s ISP product line, welcome to contact us.
For more information about iCatch’s products, please visit www.icatchtek.com
For sales contact, please leave your message at www.icatchtek.com/Requests
Reference
[1] Wikipedia, Perspective control https://en.wikipedia.org/wiki/Perspective_control
[2] Wikipedia, Illuminance https://en.wikipedia.org/wiki/Illuminance
[3] Sony, Key technology for addressing sensing application in automotive https://www.youtube.com/watch?v=1rkQFOxlMZg&feature=share
[4] EETimes, ADAS Front Camera: Demystifying Resolution and Frame-Rate