News

Vehicle in-cabin monitoring system is a device or system that can monitor and protect the occupants (drivers and passengers) inside the vehicle through safety warnings. It includes two subsystems that can operate separately: driver monitoring system (DMS) and occupancy monitoring system (OMS).

 

 

DMS can monitor and detect the driver's behavior and physical state that may cause driving mistakes, such as fatigue, doze, smoking or eating and so on. It will send out alarms promptly to remind the driver to avoid accidents caused by negligence, and protect the safety of other road users. OMS can avoid children, pets, or objects left in car to ensure the passengers’ safety and enhance the ride experience.

This article will introduce the regulatory requirements of the in-vehicle safety monitoring system and then summarize the functional requirements and implementation technology of DMS and OMS.
 

1. European Union takes the lead in legislation to list DMS as a necessary equipment for new cars
Most traffic accidents can be avoided. According to an international research statistics, at least 90% of vehicle traffic accidents are caused by human error. A study of NHTSA in 2016 showed that the proportion of traffic accidents caused by human negligence is between 94% and 96%^[1]. Therefore, if we can avoid or reduce the occurrence of human error, the probability of road traffic accidents as well as the threat and impact on other pedestrians will also be reduced.

According to the European Commission^[2], some observations shows that around 25,000 people die in traffic accidents in Europe every year, most of which are due to human error. In order to improve this situation, EU legislation in 2019 requires various vehicles to be equipped with driving assistance warning systems according to vehicle categories^[3], among which DMS is mandatory for all vehicles. This requirement applies to new vehicles manufactured after July 2022 and all models manufactured after July 2024.

The United States is promoting the Hot Cars Act bill^[4], which aims to alert drivers that children are left in sweltering vehicles to protect kids from hot car deaths. It can also be applied to the detection of pets or passengers. This is very important regarding to improving the safety of passengers.
 

2. The requirements of driver monitoring system
The driver monitoring system can be used to detect the following risk.
1) Drowsiness: such as tiredness, yawning, drooping eyelids, dozing off with closed eyes, etc.
2) Attentiveness: such as glancing left or right or not looking ahead while driving.
3) Distraction: such as smoking, drinking, eating, or using a mobile phone while driving, etc.

 

Some traditional methods monitor the driver's state by detecting the speed change, the steering wheel rotation angle and the braking force. But these cannot confirm the real state of the driver. Modern driver monitoring systems mainly use image processing to detect the eye opening and blinking frequency to determine whether the driver is tired or drowsy.

In order to make sure the driver's attention is focused on the front of the car, it is necessary to judge the driver’s attentiveness by detecting the position of the driver's pupils and checking the front alignment to the driver’s line of sight.

Detecting distraction, such as smoking, drinking, using mobile phones while driving and so on requires advanced recognition technology. The detection rate of traditional computer vision (CV) methods is limited while the artificial intelligence deep learning algorithms using convolutional neural networks (CNN) can greatly improve the recognition accuracy.

Once detected that the driver is drowsy, inattentive or distracted to do other things, the driver monitoring system will alert the driver in time to ensure driving safety.
 

3. The realization technology of driver monitoring system
Currently, most of the tier-1 suppliers in globe auto parts manufacturing industry have developed products and technologies for driver monitoring systems. For example, Bosch's driver drowsiness detection system enrolls a camera to recognize driver's eyes opening and closing size, the frequency of blinking, and the line of sight movement. It also embeds temperature and wireless sensors to measure the driver's body temperature, heart rate and respiration rate, and can be used to determine whether the driver's physiological condition is suitable for driving.

In order to capture high quality images at night or in poor lighting conditions to clearly monitor the driver's behavior, most driver monitoring systems use IR cameras with IR LEDs, such as products of Valeo, Denso Corporation, and Hyundai Mobis.

 

 
iCatch's intelligent automotive ISP solution Ci1 can process both RGB and IR images at the same time. The embedded neural processing engine in Ci1 chip can also realize driver distraction recognition, which is very suitable for use in the driver monitoring system. An international AI technology leader selected iCatch Ci1 to develop its new smart driving monitoring camera solution.
 

4. The requirements and realization technology of the occupancy monitoring system
The occupancy monitoring system is mainly to determine whether there is something or someone left in the car and to detect the presence of an object or person in a seat. Drivers will receive an instant alert notification if any children or pets are left behind in the vehicle.

Simple occupant detection uses mechanical weight sensor. mmWave radar is for vital signs monitoring. Image processing technology is necessary for analyzing passenger’s behavior and confirm their safety.

The advanced functions of occupancy monitoring system are mainly used to improve ride experience.
1) Passengers' emotion detection, which can be used to change and adjust the music or ambient lighting in the vehicle.
2) Passengers' gesture recognition, which can be used for air conditioning control, track switching and volume control of the entertainment system in the cabin.
3) Passengers' activity detection, which can be used to detect and warn for quarrel or violence in the car to ensure the passengers remain safe.

 

At present, there are only a few manufacturers that provide occupancy monitoring system solution, such as tier-1 automotive suppliers Visteon and Xperi in the United States. Visteon's real-time occupant monitoring solution incorporate thermal image processing technology which can detect the liveness position on the seat, and alert drivers in time while identifying children or pets. Xperi's in-cabin monitoring systems uses dynamic visual sensing technology to monitor the movements of the driver and passengers simultaneously, and issue alerts respectively.

 

iCatch is also developing a new generation of intelligent image processing system chip, which supports not only the general RGB sensor and IR sensor but also the dynamic vision sensor. It can be used to develop a new generation of occupancy monitoring systems to monitor the activities of the driver and passengers inside the vehicle.

 

5. Conclusion
In order to reduce Road traffic injuries and ensure the safety of children in the car, driver monitoring system and occupant monitoring system are necessary safety protecting features. Europe has already legislated to require new vehicles launched in the future to be equipped with driver drowsiness and attention detection systems to reduce road accidents caused by human error. The Unite States is also promoting for legislation to require vehicles to install alarm devices to prevent children from being left in cars. It is obviously that in-cabin monitoring system is a must-have feature for future cars. As the major tier-1 automotive suppliers have launched in-vehicle and driver monitoring system solutions in Europe, the United States, Japan, and South Korea, iCatch is also actively investing in the development of technologies and product solutions related to occupant and driver monitoring systems to support a wide range of applications in the future.
 

Reference
[1] Pittsburgh Car Accident Law Firm, https://www.cbmclaw/auto-accident/
[2] Road Safety: Commission welcomes agreement on new EU rules to help save lives. https://ec.europa.eu/commission/presscorner/detail/en/IP_19_1793
[3] Regulation (EU) 2019/2144 of the European Parliament and of the Council.
[4] H.R.3164 – Hot Cars Act of 2021, https://www.congress.gov/bill/117th-congress/house-bill/3164/text