REWeather: A Unified Detection Framework for Automatic Driving Images Restoration and Enhancement in Adverse Weather

Abstract

Recently, with the rapid development of autonomous driving technology, it prompts the vehicle detection technology to continuously improve its accuracy, stability and reliability to better meet the needs of self-driving. However, due to the interference of adverse factors in adverse weather, the decrease of detection accuracy of autonomous vehicle is led to the phenomenon of missing and wrong detection, which has a serious impact on the safety of autonomous vehicles. Therefore, we propose REWeather to solve such problems of autonomous vehicles in multiple adverse weather conditions. Firstly, to classify the types of adverse weather, distinguishing among foggy, rainy and snowy weather, Broad Learning System (BLS) which is simple and efficient is used in REWeather. Due to the impact of these adverse weathers on sensors, simple dark channel and guided filtering methods is used to preprocess foggy and rainy images, respectively. Then, we put the processed images into the Real-Enhanced Super-Resolution Generative Adversarial Networks (Real-ESRGAN) for further denoising and enhancing the details of detected objects, enabling the sensor to recognize other targets on the road faster and better in adverse weather. To ensure the best detection results, we also use latest Realtime Detection Transformer (RT-DETR) as the detector to validate our work and the final model is deployed on the edge device. Moreover, we use several public datasets and our own collected data to make a real world dataset containing a variety of adverse weathers to train and test our proposed framework, which makes it closer to the real situation. The results demonstrate that our framework achieves a 3.8% improvement in mAP, significantly enhancing the detection capability of autonomous vehicles under adverse weather conditions.