This repository contains the PyTorch code for our paper "Language-driven All-in-one Adverse Weather Removal" by Hao Yang, Liyuan Pan, Yan Yang and Wei Liang.
The code will come soon!
All-in-one (AiO) frameworks restore various adverse weather degradations with a single set of networks jointly. To handle various weather conditions, an AiO framework is expected to adaptively learn weather-specific knowledge for different degradations and shared knowledge for common patterns. However, existing methods: 1) rely on extra supervision signals, which are usually unknown in real-world applications; 2) employ fixed network structures, which restrict the diversity of weather-specific knowledge. In this paper, we propose a Language-driven Restoration framework (LDR) to alleviate the aforementioned issues. First, we leverage the power of pre-trained vision-language (PVL) models to enrich the diversity of weather-specific knowledge by reasoning about the occurrence, type, and severity of degradation, generating description-based degradation priors. Then, with the guidance of degradation prior, we sparsely select restoration experts from a candidate list dynamically based on a Mixture-of-Experts (MoE) structure. This enables us to adaptively learn the weather-specific and shared knowledge to handle various weather conditions (e.g., unknown or mixed weather). Experiments on extensive restoration scenarios show our superior performance .
Please refer to requirements.txt.
python main.py@article{yang2023language,
title={Language-driven All-in-one Adverse Weather Removal},
author={Yang, Hao and Pan, Liyuan and Yang, Yan and Liang, Wei},
journal={arXiv preprint arXiv:2312.01381},
year={2023}
}
The code is borrowed from the following repositories, thanks for sharing.