## Tensorflow Fourier Feature Mapping Networks

Tensorflow 2.0 implementation of Fourier Feature Mapping networks from the paper Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains.

## Installation

- Pip install

`$ pip install --upgrade tf_fourier_features`

- Pip install (test support)

`$ pip install --upgrade tf_fourier_features[tests]`

## Usage

```
from tf_fourier_features import FourierFeatureProjection
from tf_fourier_features import FourierFeatureMLP
# You should use FourierFeatureProjection right after the input layer.
ip = tf.keras.layers.Input(shape=[2])
x = FourierFeatureProjection(gaussian_projection = 256, gaussian_scale = 1.0)(ip)
x = tf.keras.layers.Dense(256, activation='relu')(x)
x = tf.keras.layers.Dense(3, activation='sigmoid')(x)
model = tf.keras.Model(inputs=ip, outputs=x)
# Or directly use the model class to build a multi layer Fourier Feature Mapping Network
model = FourierFeatureMLP(units=256, final_units=3, final_activation='sigmoid', num_layers=4,
gaussian_projection=256, gaussian_scale=10.0)
```

## Results on Image Inpainting task

A partial implementation of the image inpainting task is available as the `train_inpainting_fourier.py`

and `eval_inpainting_fourier.py`

scripts inside the `scripts`

directory.

Weight files are made available in the repository under the `Release`

tab of the project. Extract the weights and place the `checkpoints`

folder at the scripts directory.

These weights generates the following output after 2000 epochs of training with batch size 8192 while using only 10% of the available pixels in the image during training phase.

If we train for using only 20% of the available pixels in the image during training phase -

If we train for using only 30% of the available pixels in the image during training phase - .

## Results on Multi Image Inpainting

It is possible to encode multiple images into a single network by using an augmented input latent vector. The latent vector can be of any size (here, set to 8) and conditions the model to predict the pixel of a certain image, even when given the same (x,y) coordinate for different images.

The code to train this type of model is available in the `scripts`

directory - `train_multi_inpainting_fourier.py`

and `eval_multi_inpainting_fourier.py`

.

Below are the result of encoding 3 images in to a single model with 260K parameters, trained for 2000 epochs using 30% of the pixels per image as input training data at 800x800 pixel resolution. This is equivalent to training the model with 576K training samples.

## Citation

```
@misc{tancik2020fourier,
title={Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains},
author={Matthew Tancik and Pratul P. Srinivasan and Ben Mildenhall and Sara Fridovich-Keil and Nithin Raghavan and Utkarsh Singhal and Ravi Ramamoorthi and Jonathan T. Barron and Ren Ng},
year={2020},
eprint={2006.10739},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```

## Requirements

- Tensorflow 2.0+
- Matplotlib to visualize eval result