HuHaigen

A-Channel-Spatial-Hybrid-Attention-Mechanism-using-Channel-Weight-Transfer-Strategy

Attention is one of the most valuable breakthroughs in the deep learning community, and how to effectively utilize the attention information of channel and spatial is still one of the hot research topics. In this work, we integrate the advantages of channel and spatial mechanism to propose a Channel-Spatial hybrid Attention Module (CSHAM). Specifically, max-average fusion Channel Attention Module and Spatial Attention Neighbor Enhancement Module are firstly proposed, respectively. Then the connection between the two modules is analyzed and designed, and an alternate connection strategy with the transformation of channel weights is proposed. The key idea is to repeatedly use the channel weight information generated by the channel attention module, and to reduce the negative impact of the network complexity caused by the addition of the attention mechanism. Finally, a series of comparison experiments are conducted on CIFAR100 and Caltech-101 based on various backbone models. The results show that the proposed methods can obtain the best Top-1 performance among the existing popular methods, and can rise by nearly 1% in accuracy while basically maintaining the parameters and FLOPs.

Adaptively-Customizing-Activation-Functions

To enhance the nonlinearity of neural networks and increase their mapping abilities between the inputs and response variables, activation functions play a crucial role to model more complex relationships and patterns in the data. In this work, a novel methodology is proposed to adaptively customize activation functions only by adding very few parameters to the traditional activation functions such as Sigmoid, Tanh, and ReLU. To verify the effectiveness of the proposed methodology, some theoretical and experimental analysis on accelerating the convergence and improving the performance is presented, and a series of experiments are conducted based on various network models (such as AlexNet, VGGNet, GoogLeNet, ResNet and DenseNet), and various datasets (such as CIFAR10, CIFAR100, miniImageNet, PASCAL VOC and COCO) . To further verify the validity and suitability in various optimization strategies and usage scenarios, some comparison experiments are also implemented among different optimization strategies (such as SGD, Momentum, AdaGrad, AdaDelta and ADAM) and different recognition tasks like classification and detection. The results show that the proposed methodology is very simple but with significant performance in convergence speed, precision and generalization, and it can surpass other popular methods like ReLU and adaptive functions like Swish in almost all experiments in terms of overall performance.

CorGrad

Official Implementation of CorGrad (ECAI 2024)

COVID-19-Lung-Infection-Segmentation

Due to the irregular shapes,various sizes and indistinguishable boundaries between the normal and infected tissues, it is still a challenging task to accurately segment the infected lesions of COVID-19 on CT images. In this paper, a novel segmentation scheme is proposed for the infections of COVID-19 by enhancing supervised information and fusing multi-scale feature maps of different levels based on the encoder-decoder architecture. To this end, a deep collaborative supervision (Co-supervision) scheme is proposed to guide the network learning the features of edges and semantics. More specifically, an Edge Supervised Module (ESM) is firstly designed to highlight low-level boundary features by incorporating the edge supervised information into the initial stage of down-sampling. Meanwhile, an Auxiliary Semantic Supervised Module (ASSM) is proposed to strengthen high-level semantic information by integrating mask supervised information into the later stage. Then an Attention Fusion Module (AFM) is developed to fuse multiple scale feature maps of different levels by using an attention mechanism to reduce the semantic gaps between high-level and low-level feature maps. Finally, the effectiveness of the proposed scheme is demonstrated on four various COVID-19 CT datasets. The results show that the proposed three modules are all promising. Based on the baseline (ResUnet), using ESM, ASSM, or AFM alone can respectively increase Dice metric by 1.12%, 1.95%,1.63% in our dataset, while the integration by incorporating three models together can rise 3.97%. Compared with the existing approaches in various datasets, the proposed method can obtain better segmentation performance in some main metrics, and can achieve the best generalization and comprehensive performance.

CTI-UNet

[ICIP 2023] CTI-Unet: Hybrid Local Features and Global Representations Efficiently

Cut-Stitch

HDConv

Joint-Feature-Learning-for-Cell-Segmentation

[BIBM 2022] Joint Feature Learning for Cell Segmentation Based on Multi-scale Convolutional U-Net

Mask-R-DHCNN

TDRConv-Exploring-the-Trade-off-Between-Feature-Diversity-and-Redundancy-for-a-Compact-CNN-Module

HuHaigen/COVID-19-Lung-Infection-Segmentation

Due to the irregular shapes,various sizes and indistinguishable boundaries between the normal and infected tissues, it is still a challenging task to accurately segment the infected lesions of COVID-19 on CT images. In this paper, a novel segmentation scheme is proposed for the infections of COVID-19 by enhancing supervised information and fusing multi-scale feature maps of different levels based on the encoder-decoder architecture. To this end, a deep collaborative supervision (Co-supervision) scheme is proposed to guide the network learning the features of edges and semantics. More specifically, an Edge Supervised Module (ESM) is firstly designed to highlight low-level boundary features by incorporating the edge supervised information into the initial stage of down-sampling. Meanwhile, an Auxiliary Semantic Supervised Module (ASSM) is proposed to strengthen high-level semantic information by integrating mask supervised information into the later stage. Then an Attention Fusion Module (AFM) is developed to fuse multiple scale feature maps of different levels by using an attention mechanism to reduce the semantic gaps between high-level and low-level feature maps. Finally, the effectiveness of the proposed scheme is demonstrated on four various COVID-19 CT datasets. The results show that the proposed three modules are all promising. Based on the baseline (ResUnet), using ESM, ASSM, or AFM alone can respectively increase Dice metric by 1.12%, 1.95%,1.63% in our dataset, while the integration by incorporating three models together can rise 3.97%. Compared with the existing approaches in various datasets, the proposed method can obtain better segmentation performance in some main metrics, and can achieve the best generalization and comprehensive performance.

HuHaigen/Adaptively-Customizing-Activation-Functions

To enhance the nonlinearity of neural networks and increase their mapping abilities between the inputs and response variables, activation functions play a crucial role to model more complex relationships and patterns in the data. In this work, a novel methodology is proposed to adaptively customize activation functions only by adding very few parameters to the traditional activation functions such as Sigmoid, Tanh, and ReLU. To verify the effectiveness of the proposed methodology, some theoretical and experimental analysis on accelerating the convergence and improving the performance is presented, and a series of experiments are conducted based on various network models (such as AlexNet, VGGNet, GoogLeNet, ResNet and DenseNet), and various datasets (such as CIFAR10, CIFAR100, miniImageNet, PASCAL VOC and COCO) . To further verify the validity and suitability in various optimization strategies and usage scenarios, some comparison experiments are also implemented among different optimization strategies (such as SGD, Momentum, AdaGrad, AdaDelta and ADAM) and different recognition tasks like classification and detection. The results show that the proposed methodology is very simple but with significant performance in convergence speed, precision and generalization, and it can surpass other popular methods like ReLU and adaptive functions like Swish in almost all experiments in terms of overall performance.

HuHaigen/A-Channel-Spatial-Hybrid-Attention-Mechanism-using-Channel-Weight-Transfer-Strategy

Attention is one of the most valuable breakthroughs in the deep learning community, and how to effectively utilize the attention information of channel and spatial is still one of the hot research topics. In this work, we integrate the advantages of channel and spatial mechanism to propose a Channel-Spatial hybrid Attention Module (CSHAM). Specifically, max-average fusion Channel Attention Module and Spatial Attention Neighbor Enhancement Module are firstly proposed, respectively. Then the connection between the two modules is analyzed and designed, and an alternate connection strategy with the transformation of channel weights is proposed. The key idea is to repeatedly use the channel weight information generated by the channel attention module, and to reduce the negative impact of the network complexity caused by the addition of the attention mechanism. Finally, a series of comparison experiments are conducted on CIFAR100 and Caltech-101 based on various backbone models. The results show that the proposed methods can obtain the best Top-1 performance among the existing popular methods, and can rise by nearly 1% in accuracy while basically maintaining the parameters and FLOPs.

HuHaigen/CTI-UNet

[ICIP 2023] CTI-Unet: Hybrid Local Features and Global Representations Efficiently

HuHaigen/TDRConv-Exploring-the-Trade-off-Between-Feature-Diversity-and-Redundancy-for-a-Compact-CNN-Module

HuHaigen/HDConv

HuHaigen/Joint-Feature-Learning-for-Cell-Segmentation

[BIBM 2022] Joint Feature Learning for Cell Segmentation Based on Multi-scale Convolutional U-Net

HuHaigen/CorGrad

Official Implementation of CorGrad (ECAI 2024)

HuHaigen/Cut-Stitch

HuHaigen/Mask-R-DHCNN

HuHaigen/SAMDConv

[PRCV 2023] SAMDConv: Spatially Adaptive Multi-scale Dilated Convolution

HuHaigen/SGT