High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks
Fig. 2: Eight testing samples selected from CIFAR10 that help explain that CNN can capture the high-frequency image: the model (ResNet18) correctly predicts the original image (1st column in each panel) and the high frequency reconstructed image (3rd column in each panel), but incorrectly predict the low-frequency reconstructed image (2nd column in each panel). The prediction confidences are also shown. Details are in the paper.
Other Discussions in Paper (click to expand)
- Trade-off between accuracy and robustness (Section 3)
- Rethinking data before rethinking generalization (Section 4)
- Re-evaluate the heuristics (BatchNorm seems to promote high-frequency information) (Section 5)
- Adversarially robust models tend to filter out high-frequency components (Section 6)
- Similar phenomena are observed beyond image classification (Section 7)
- Pipeline to replicate our results on CIFAR10 data
Before using the code
- Install the main dependency: TensorFlow 1.x and Foolbox
- Read the main script. We recommend users to skim through the script befoure usage. We use "todo" to highlight the parts that may require attention.
- Generate data: one can generate the data with utility/frequencyHelper.py
The results were generated by the TensorFlow code as shared here, but for the friends who prefer PyTorch, Xindi has nicely created some codes to help you start: PyTorch Implementation