This repo contains all the code required to replicate the paper:
The workflow is:
- Generate the training data (simulation takes a few days for the 3D dataset).
- Download the models and voxelize them.
- Run mantaflow to generate fluid data.
- Train the network in Torch7 (training takes about 1-2 days).
- Train and validate the model.
- (optional) Run 3D example script to create videos from the paper.
- (optional) Run 2D real-time demo.
The limitations of the current system are outlined at the end of this doc. Please read it before even considering integrating our code.
Note: This is not an official Google product.
UPDATES / NEWS:
Oct 14 2019
- Added a pre-trained 2D model to the repo (for use with the realtime demo). Use this if you just want to visualize some output running in realtime (skipping data creation, model training, etc). See realtime demo instructions below.
Oct 13 2019
- Verified instructions were still up to date. torch7 does not work with CUDA SDK newer than 9.2 (so this is what you must use). If you're using CUDA 10.0 or 10.1, you need to downgrade. For CUDNN make sure you install v7.6.4 for CUDA 9.2. Additionally, CUDA 9.2 does not support gcc>7 AND there seems to be an issue with half float support at cutorch head. I had to install torch using:
See here for more cuda 9.2 debug help. In addition to get cudnn to work with v7.6.4 I used:
./clean.sh export TORCH_NVCC_FLAGS="-D__CUDA_NO_HALF_OPERATORS__" export CC=/usr/bin/gcc-7 export CXX=/usr/bin/g++-7 ./install.sh'
As discussed here.
git clone https://github.com/soumith/cudnn.torch.git -b R7 && cd cudnn.torch && luarocks make cudnn-scm-1.rockspec
- Fixed a few compile issues with tfluids: missing vector and changes to THCudaTensor_norm as pointed out in issue 10.
Feb 6 2017
- Huge refactoring and bug-fix update (too many to mention here!).
- Switched data everywhere to MAC-Grid (instead of central sampling).
- Numerous improvements to Convnet model (some departure from arxiv paper, paper revision coming in February/March).
tfluidsnow has a
third_partysub-library. This is essentially a port of some of Manta's simulator code (to torch + CUDA). Note that it is released under the GNU GPL V3 license (as per Manta's licensing).
- GPU PCG (using NVidia's cusparse library) and Jacobi methods added as baseline methods.
Jan 6 2017
- Lots of updates to model and training code.
- Added model of Yang et al. "Data-driven projection method in fluid simulation" as a baseline comparison.
- Changed model defaults (no more pooling, smaller network no more p loss term).
- Improved programmability of the model from the command line.
- Added additional scripts to plot debug data and performance vs. epoch.
Dec 21 2016
- Refactor of data processing code.
- Batch creation is now asynchronous and parallel (to hide file IO latency). Results in slight speed up in training for most systems, and significant speedup for disk IO limited systems (i.e. when files are on a DFS).
- Data cache paths are now relative, so that cache data can be moved around.
- Implemented advection in CUDA; entire simulation.lua loop is now on the GPU. Significant speedup for 3D models (both training and eval) and slight speedup for 2D models.
- Numerous bug fixes and cleanup.
#0. Clone this repo:
git clone [email protected]:google/FluidNet.git
#1. Generating the data
CREATING VOXELIZED MODELS
We use a subset of the NTU 3D Model Database models (http://3d.csie.ntu.edu.tw/~dynamic/database/). Please download the model files:
cd FluidNet/voxelizer mkdir objs cd objs wget http://3d.csie.ntu.edu.tw/~dynamic/database/NTU3D.v1_0-999.zip # wget https://cs.nyu.edu/~schlacht/NTU3D.v1_0-999.zip # Alternate download location. unzip NTU3D.v1_0-999.zip wget https://www.dropbox.com/sh/5f3t9abmzu8fbfx/AAAkzW9JkkDshyzuFV0fAIL3a/bunny.capped.obj
Next we use the binvox library (http://www.patrickmin.com/binvox/) to create voxelized representations of the NTU models. Download the executable for your platform and put the
binvox executable file in
FluidNet/voxelizer. Then run our script:
cd FluidNet/voxelizer chmod u+x binvox python generate_binvox_files.py
Note: some users have reported that they need to install
sudo apt-get install lib3ds-1-3
OPTIONAL: You can view the output by using the viewvox utility (http://www.patrickmin.com/viewvox/). Put the viewvox executable in the
FluidNet/voxelizer/voxels directory, then:
cd FluidNet/voxelizer/voxels chmod u+x viewvox ./viewvox -ki bunny.capped_32.binvox
The first step is to download the custom manta fork.
cd FluidNet/ git clone [email protected]:kristofe/manta.git
Next, you must build mantaflow using the cmake system.
cd FluidNet/manta mkdir build cd build sudo apt-get install doxygen libglu1-mesa-dev mesa-common-dev qtdeclarative5-dev qml-module-qtquick-controls cmake .. -DGUI='OFF' make -j8
For the above cmake command setting
-DGUI='ON' will slow down simulation but you can view the flow fields. You will now have a binary called manta in the build directory.
GENERATING TRAINING DATA
Install matlabnoise (https://github.com/jonathantompson/matlabnoise) to the SAME path that FluidNet is in. i.e. the directory structure should be:
To install matlabnoise (with python bindings):
sudo apt-get install python3.5-dev sudo apt-get install swig git clone [email protected]:jonathantompson/matlabnoise.git cd matlabnoise sh compile_python3.5_unix.sh sudo apt-get install python3-matplotlib python3.5 test_python.py
Now you're ready to generate the training data. Make sure the directory
data/datasets/output_current exists. For the 3D training data run:
cd FluidNet/manta/build ./manta ../scenes/_trainingData.py --dim 3 --addModelGeometry True --addSphereGeometry True
For the 2D data run:
cd FluidNet/manta/build ./manta ../scenes/_trainingData.py --dim 2 --addModelGeometry True --addSphereGeometry True
#2. Training the model
RUNNING TORCH7 TRAINING
We assume that Torch7 is installed, otherwise follow the instructions here. We use CUDA 9.2 SDK and the standard distribution (pulled on Oct 13th 2019). As of 10/2019, LUA52 is broken, so now use the LUAJIT version (this is the default anyway). Lastly, there are some install notes worth reading in the Oct 2019 update above.
After install torch, compile tfluids: this is our custom CUDA & C++ library that implements a large number of the modules used in the paper:
sudo apt-get install freeglut3-dev sudo apt-get install libxmu-dev libxi-dev cd FluidNet/torch/tfluids luarocks make tfluids-1-00.rockspec
Note #1: some users are reporting that you need to explicitly install findCUDA for tfluids to compile properly with CUDA 7.5 and above.
luarocks install findCUDA
Note #2: If THC.h cannot be found during compilation, make sure cutorch is installed properly (check that torch7 install.sh could find the CUDA SDK).
All training related code is in
torch/ directory. To train a model on 3D data:
cd FluidNet/torch qlua fluid_net_train.lua -gpu 1 -dataset output_current_3d_model_sphere -modelFilename myModel3D
This will pull data from the directory
output_current_3d_model_sphere and dump the model to
myModel3D. To train a 2D model:
cd FluidNet/torch qlua fluid_net_train.lua -gpu 1 -dataset output_current_model_sphere -modelFilename myModel2D
At any point during the training sim you can plot test and training set loss values using the Matlab script
You can control any model or training config parameters from the command line. If you need to define nested variables the syntax is:
qlua fluid_net_train.lua -new_model.num_banks 2
i.e nested variables are
. separated. You can print a list of possible config variables using:
qlua fluid_net_train.lua --help
Note: the first time the data is loaded from the manta output, it is cached to the torch/data/ directory. So if you need to reload new data (because you altered the dataset) then delete the cache files (
RUNNING THE REAL-TIME DEMO
For 2D models only! To run the interactive demo firstly compile LuaGL:
git clone [email protected]:kristofe/LuaGL.git cd LuaGL luarocks make luagl-1-02.rockspec
Then run the simulator:
cd FluidNet/torch qlua -lenv fluid_net_2d_demo.lua -gpu 1 -dataset output_current_model_sphere -modelFilename myModel2D
The command line output will print a list of possible key and mouse strokes.
RUNNING THE 3D SIMULATIONS
To render the videos you will need to install Blender, but to just create the volumetric data no further tools are needed. First run our 3D example script (after training a 3D model):
cd FluidNet/torch qlua fluid_net_3d_sim.lua -gpu 1 -loadVoxelModel none -modelFilename myModel3D
To control which scene is loaded, use the
loadVoxelModel="none"|"arc"|"bunny"```. This will dump a large amount of volumetric data to the file FluidNet/blender/<mushroom_cloud|bunny|arch>_render/output_density.vbox``.
Now that the fluid simulation has run, you can render the frames in Blender. Note that rendering takes a few hours, while the 3D simulation is fast (with a lot of time spent dumping the results to disk). An implementation of a real-time 3D fluid render is outside the scope of this work. In addition, self-advection of the velocity field is currently carried out on the CPU and so is the slowest part of our simulator (a CUDA implementation is future work).
For the mushroom cloud render, open
FluidNet/blender/MushroomRender.blend. Next we need to re-attach the data file (because blender caches full file paths which will now be wrong). Click on the "Smoke" object in the "Outliner" window (default top right). Click on the "Texture" icon in the "Properties" window (default bottom right), it's the one that looks like a textured Square. Scroll down to "Voxel Data" -> "Source Path:" and click the file icon. Point the file path to
/path/to/FluidNet/blender/mushroom_cloud_render/density_output.vbox. Next, click either the file menu "Render" -> "Render Image", or "Render Animation". By default the render output goes to
/tmp/. You can also scrub through the frame index on the time-line window (default bottom) to click a frame you want then render just that frame.
The above instructions also apply to the bunny and arch examples. Note: you might need to re-center the model depending on your version of binvox (older versions of binvox placed the voxelided model in a different location). If this is the case, then click on the "GEOM" object in the "Outliner" window. Click the eye and camera icons (so they are no longer greyed out). Then press "Shift-Z" to turn on the geometry render preview. Now that you can see the geometry and model, you can manually align the two so they overlap.
3. Limitations of the current system
While this codebase is relatively self-contained and full-featured, it is not a "ready-to-ship" fluid simluator. Rather it is a proof of concept and research platform only. If you are interested in integrating our network into an existing system feel free to reach out ([email protected]) and we will do our best to answer your questions.
The entire simulation loop is not optimized; however it is fast enough for real-time applications, where good GPU resources are available (i.e. NVidia 1080 or Titan).
Our example boundary condition code is very rudimentary. However, we support the same cell types as Manta (in-flow, empty, occupied, etc), so more complicated boundary conditions can be created. One potential limitation is that the setWallBcs codepath assumes zero velocity occupiers (like Manta does). However, it would be an easy extension to allows internal occupied voxels to have non-zero velocity.
We do not have a real-time 3D fluid render. We use an offline render instead. For our 2D "renderer", we simply display the RGB density field to screen and visualize the velocity vectors. It is very rudimentary. Incorporating an open-source 3D fluid render is future work.
The only external forces that are supported are vorticity confinement and buoyancy. Viscosity and gravity are not supported (but could be added easily).
We have unit tests (including FD gradient checks) for all custom torch modules.
The two main test scripts we do have are:
cd FluidNet/ qlua lib/modules/test_ALL_MODULES.lua
and (this one requires us to generate data from manta first):
cd FluidsNet/manta/build ./manta ../scenes/_testData.py cd ../../torch/tfluids qlua -ltfluids -e "tfluids.test()"
You should run these first if you ever get into trouble training or running the model.