Deep Learning with Keras and Tensorflow
Author: Valerio Maggio
PostDoc Data Scientist @ FBK/MPBA
Contacts:
@leriomaggio  +ValerioMaggio 
valeriomaggio  vmaggio_at_fbk_dot_eu 
git clone https://github.com/leriomaggio/deeplearningkerastensorflow.git

Part I: Introduction

Intro to Artificial Neural Networks
 Perceptron and MLP
 naive purePython implementation
 fast forward, sgd, backprop

Introduction to Deep Learning Frameworks
 Intro to Theano
 Intro to Tensorflow
 Intro to Keras
 Overview and main features
 Overview of the
core
layers  MultiLayer Perceptron and Fully Connected
 Examples with
keras.models.Sequential
andDense
 Examples with
 Keras Backend


Part II: **Supervised Learning **

Fully Connected Networks and Embeddings
 Intro to MNIST Dataset
 Hidden Leayer Representation and Embeddings

Convolutional Neural Networks

meaning of convolutional filters
 examples from ImageNet

Visualising ConvNets

Advanced CNN
 Dropout
 MaxPooling
 Batch Normalisation

HandsOn: MNIST Dataset
 FC and MNIST
 CNN and MNIST

Deep Convolutiona Neural Networks with Keras (ref:
keras.applications
) VGG16
 VGG19
 ResNet50


Transfer Learning and FineTuning

Hyperparameters Optimisation


Part III: Unsupervised Learning
 AutoEncoders and Embeddings
 AutoEncoders and MNIST
 word2vec and doc2vec (gensim) with
keras.datasets
 word2vec and CNN
 word2vec and doc2vec (gensim) with

Part IV: Recurrent Neural Networks
 Recurrent Neural Network in Keras
SimpleRNN
,LSTM
,GRU
 LSTM for Sentence Generation
 Recurrent Neural Network in Keras

PartV: Additional Materials:
 Custom Layers in Keras
 Multi modal Network Topologies with Keras
Requirements
This tutorial requires the following packages:

Python version 3.5
 Python 3.4 should be fine as well
 likely Python 2.7 would be also fine, but who knows? :P

numpy
version 1.10 or later: http://www.numpy.org/ 
scipy
version 0.16 or later: http://www.scipy.org/ 
matplotlib
version 1.4 or later: http://matplotlib.org/ 
pandas
version 0.16 or later: http://pandas.pydata.org 
scikitlearn
version 0.15 or later: http://scikitlearn.org 
keras
version 2.0 or later: http://keras.io 
tensorflow
version 1.0 or later: https://www.tensorflow.org 
ipython
/jupyter
version 4.0 or later, with notebook support
(Optional but recommended):
pyyaml
hdf5
andh5py
(required if you use model saving/loading functions in keras) NVIDIA cuDNN if you have NVIDIA GPUs on your machines. https://developer.nvidia.com/rdp/cudnndownload
The easiest way to get (most) these is to use an allinone installer such as Anaconda from Continuum. These are available for multiple architectures.
Python Version
I'm currently running this tutorial with Python 3 on Anaconda
!python version
Python 3.5.2
Setting the Environment
In this repository, files to recreate virtual env with conda
are provided for Linux and OSX systems, namely deeplearning.yml
and deeplearningosx.yml
, respectively.
To recreate the virtual environments (on Linux, for example):
conda env create f deeplearning.yml
For OSX, just change the filename, accordingly.
Notes about Installing Theano with GPU support
NOTE: Read this section only if after pip installing theano
, it raises error in enabling the GPU support!
Since version 0.9
Theano introduced the libgpuarray
in the stable release (it was previously only available in the development version).
The goal of libgpuarray
is (from the documentation) make a common GPU ndarray (n dimensions array) that can be reused by all projects that is as future proof as possible, while keeping it easy to use for simple need/quick test.
Here are some useful tips (hopefully) I came up with to properly install and configure theano
on (Ubuntu) Linux with GPU support:
 [If you're using Anaconda]
conda install theano pygpu
should be just fine!
Sometimes it is suggested to install pygpu
using the condaforge
channel:
conda install c condaforge pygpu
 [Works with both Anaconda Python or Official CPython]

Install
libgpuarray
from source: Stepbystep install libgpuarray user library 
Then, install
pygpu
from source: (in the same source folder)python setup.py build && python setup.py install

pip install theano
.
After Theano is installed:
echo "[global]
device = cuda
floatX = float32
[lib]
cnmem = 1.0" > ~/.theanorc
Installing Tensorflow
To date tensorflow
comes in two different packages, namely tensorflow
and tensorflowgpu
, whether you want to install the framework with CPUonly or GPU support, respectively.
For this reason, tensorflow
has not been included in the conda envs and has to be installed separately.
Tensorflow for CPU only:
pip install tensorflow
Tensorflow with GPU support:
pip install tensorflowgpu
Note: NVIDIA Drivers and CuDNN must be installed and configured before hand. Please refer to the official Tensorflow documentation for further details.
Important Note:
All the code provided+ in this tutorial can run even if tensorflow
is not installed, and so using theano
as the (default) backend!
This is exactly the power of Keras!
Therefore, installing tensorflow
is not stricly required!
+: Apart from the 1.2 Introduction to Tensorflow tutorial, of course.
Configure Keras with tensorflow
By default, Keras is configured with theano
as backend.
If you want to use tensorflow
instead, these are the simple steps to follow:
 Create the
keras.json
(if it does not exist):
touch $HOME/.keras/keras.json
 Copy the following content into the file:
{
"epsilon": 1e07,
"backend": "tensorflow",
"floatx": "float32",
"image_data_format": "channels_last"
}
 Verify it is properly configured:
!cat ~/.keras/keras.json
{
"epsilon": 1e07,
"backend": "tensorflow",
"floatx": "float32",
"image_data_format": "channels_last"
}
Test if everything is up&running
1. Check import
import numpy as np
import scipy as sp
import pandas as pd
import matplotlib.pyplot as plt
import sklearn
import keras
Using TensorFlow backend.
2. Check installed Versions
import numpy
print('numpy:', numpy.__version__)
import scipy
print('scipy:', scipy.__version__)
import matplotlib
print('matplotlib:', matplotlib.__version__)
import IPython
print('iPython:', IPython.__version__)
import sklearn
print('scikitlearn:', sklearn.__version__)
numpy: 1.11.1
scipy: 0.18.0
matplotlib: 1.5.2
iPython: 5.1.0
scikitlearn: 0.18
import keras
print('keras: ', keras.__version__)
# optional
import theano
print('Theano: ', theano.__version__)
import tensorflow as tf
print('Tensorflow: ', tf.__version__)
keras: 2.0.2
Theano: 0.9.0
Tensorflow: 1.0.1