Code of JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection(CVPR2020)

JL-DCF-pytorch

Pytorch implementation for JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection (CVPR2020) [PDF][中文版]

Requirements

• Python 3.6
  
• Pytorch 1.5.0
  
• Torchvision 0.6.1
  
• Cuda 10.0

Usage

This is the Pytorch implementation of JL-DCF. It has been trained and tested on Windows (Win10 + Cuda 10 + Python 3.6 + Pytorch 1.5), and it should also work on Linux but we didn't try.

To Train

• Download the pre-trained ImageNet backbone (resnet101 and vgg_conv1, whereas the latter already exists in the folder), and put it in the 'pretrained' folder
• Download the training dataset and modify the 'train_root' and 'train_list' in the main.py
• Set 'mode' to 'train'
• Run main.py

To Test

• Download the testing dataset and have it in the 'dataset/test/' folder
• Download the already-trained JL-DCF pytorch model and modify the 'model' to its saving path in the main.py
• Modify the 'test_folder' in the main.py to the testing results saving folder you want
• Modify the 'sal_mode' to select one testing dataset (NJU2K, NLPR, STERE, RGBD135, LFSD or SIP)
• Set 'mode' to 'test'
• Run main.py

Learning curve

The training log is saved in the 'log' folder. If you want to see the learning curve, you can get it by using: tensorboard --logdir your-log-path

Pre-trained ImageNet model for training

resnet101
vgg_conv1, password: rllb

Trained model for testing

Baidu Pan: JL-DCF-pytorch, password: jdpb
Google Drive: https://drive.google.com/open?id=12u37yz-031unDPJoKaZ0goK8BtPP-6Cj

JL-DCF-pytorch saliency maps

Baidu Pan: Saliency maps, password: 4nqr
Google Drive: https://drive.google.com/open?id=1mHMN36aI5zNt50DQBivSDyYvCQ9eeGhP

Dataset

Baidu Pan:
Training dataset (with horizontal flip), password: i4mi
Testing datadet, password: 1ju8
Google Drive:
Training dataset (with horizontal flip)
Testing datadet

Performance

Below is the performance of JL-DCF-pyotrch (Pytorch implementation). Generally, the performance of Pytorch implementation is comparable to, and even slightly better than the previous Caffe implementation reported in the paper. This is probably due to the differences between deep learning platforms. Also, due to the randomness in the training process, the obtained results will fluctuate slightly.

Citation

Please cite our paper if you find the work useful:

    @InProceedings{Fu_2020_CVPR,
    author = {Keren Fu, Deng-Ping Fan, Ge-Peng Ji, Qijun Zhao},
    title = {JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection},
    booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    pages={3052--3062},
    year = {2020}
    }

Benchmark RGB-D SOD

The complete RGB-D SOD benchmark can be found in this page
http://dpfan.net/d3netbenchmark/