Full Code of simonwsw/deep-soli for AI

Repository: simonwsw/deep-soli
Branch: master
Commit: e748041d59bb
Files: 26
Total size: 61.3 KB

Directory structure:
gitextract_2w_qnpkh/

├── .gitignore
├── LICENSE
├── README.md
├── cite.bib
├── config/
│   └── file_half.json
├── image/
│   ├── Makefile
│   ├── h5Data.cpp
│   ├── h5Data.hpp
│   ├── main.cpp
│   ├── seq.cpp
│   └── seq.hpp
├── net/
│   ├── data.lua
│   ├── imageseq.lua
│   ├── main.lua
│   ├── net.lua
│   ├── rnndata.lua
│   ├── rnntrain.lua
│   ├── seq.lua
│   ├── stat.lua
│   ├── train.lua
│   ├── uninet.lua
│   └── util.lua
└── pre/
    ├── image.py
    ├── main.py
    ├── op.py
    └── seq.py

================================================
FILE CONTENTS
================================================

================================================
FILE: .gitignore
================================================
# Mac
.DS_Store

# Matlab
*.m~
*.asv

# Vim
*~

# Python binary
*.pyc

# Temp dir
tmp

# Build and CMake
build

# openFrameworks
bin
obj


================================================
FILE: LICENSE
================================================
MIT License

Copyright (c) 2016 Saiwen Wang

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


================================================
FILE: README.md
================================================
# deep-soli

Gesture Recognition Using Neural Networks with Google's Project Soli Sensor

## Update

Dataset and trained model are now available.

## Introduction

This is the open source evaluation code base of our paper:

**Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition
in the Radio-Frequency Spectrum** <br />
Saiwen Wang, Jie Song, Jamie Lien, Poupyrev Ivan, Otmar Hilliges <br />
(link to the [paper](http://bit.ly/2ftSRcn))

Please cite the paper if you find the code useful. Thank you.
([BibTex](https://github.com/simonwsw/deep-soli/blob/master/cite.bib))

This project uses Google's [Project Soli](atap.google.com/soli) sensor.

> Soli is a new sensing technology that uses miniature radar to
> detect touchless gesture interactions.
>
> ![Soli sensor image](http://bit.ly/2fbwLYm)
>
> Soli sensor technology works by emitting electromagnetic waves in a
> broad beam. Objects within the beam scatter this energy, reflecting
> some portion back towards the radar antenna. Properties of the
> reflected signal, such as energy, time delay, and frequency shift
> capture rich information about the object’s characteristics and
> dynamics, including size, shape, orientation, material, distance,
> and velocity.

Our paper uses a light-weight end-to-end trained Convolutional Neural Networks
and Recurrent Neural Networks architecture, recognizes 11 in-air gestures
with 87% per-frame accuracy, and can perform realtime predictions at 140Hz
on commodity hardware. (link to the [paper video](http://bit.ly/2fDd9iJ))

## Pre-request

- Python 2: HDF5, OpenCV 2 interfaces for python.
- C++: HDF5, OpenCV 2, Boost
- Lua JIT and Torch 7.
- Torch 7 packages: `class`, GPU support `cunn` and `cutorch`, Matlab
  support `mattorch`, JSON support `lunajson`, Torch image library `image`
- Please note that `mattorch` is an outdated packages which is no
  longer maintained.

## Quick start

- Make image binary

```
cd image
mkdir bin
make
```

- Preprocessing (HDF5 to images):

```
python pre/main.py --op image --file [dataset folder]
--target [target image folder] --channel 4 --originsize 32 --outsize 32
```

- Preprocessing (generate mean file):

```
python pre/main.py --op mean --file [image folder]
--target [mean file name] --channel 4 --outsize 32
```

- Load model and evaluate:

```
th net/main.lua --file [image folder] --list [train/test sequence split file]
--load [model file] --inputsize 32 --inputch 4 --label 13 --datasize 32
--datach 4 --batch 16 --maxseq 40 --cuda --cudnn
```

## Dataset

- Download [dataset](https://polybox.ethz.ch/index.php/s/wG93iTUdvRU8EaT)
  (please let me know when the link doesn't work).
- Train/test split file (in JSON format) we used is stored in the repo
  `config/file_half.json`.
- The dataset contains multiple preprocessed Range-Doppler Image sequences.
  Each sequence is saved as a single HDF5 format data file. File names are
  defined as `[gesture ID]_[session ID]_[instance ID].h5`. Range-Doppler Image
  data of a specific channel can be accessed by dataset name `ch[channel ID]`.
  Label can be accessed by dataset name `label`. Range-Doppler Image
  data array has shape of `[number of frame] * 1024` (can be reshape back to 2D Range-Doppler Image to `32 * 32`)
- Simple Python code to access the data:

```python
# Demo code to extract data in python
import h5py

use_channel = 0
with h5py.File(file_name, 'r') as f:
    # Data and label are numpy arrays
    data = f['ch{}'.format(use_channel)][()]
    label = f['label'][()]
```

- Dataset session arrangement for evaluation.
  - 11 (gestures) * 25 (instances) * 10 (users) for cross user evaluation:
    session 2 (25), 3 (25), 5 (25), 6 (25), 8 (25), 9 (25), 10 (25), 11 (25),
    12 (25), 13 (25).
  - 11 (gestures) * (50 (instances) * 4 (sessions) +
    25 (instances) * 2 (sessions)) for single user
    cross session evaluation: session 0 (50), 1 (50), 4 (50), 7 (50),
    13 (25), 14 (25).
  - Please refer to the paper for the gesture collecting
    campaign details.
- The gestures are listed in the table below. Each column represents
  one gesture and we snapshot three important steps for each gestures.
  The gesture label is indicated by the number in the circle above. Please
  notice that the gesture label order is different than the paper, as
  we regroup gestures in the paper. Sequences with gesture ID 11 are
  background signals with no presence of hand.

![gesture](http://bit.ly/2fHcMRX)

## Pre-trained model

- Download [model](https://polybox.ethz.ch/index.php/s/0SEdZqkn433dbEh)
- Trained proposed model, please refer to the paper for model detail.
- Simple Lua (Torch 7) code to load the model:

```
loadFile = 'uni_image_np_50.t7'
net = torch.load(loadFile)
print(net)
```

- The model uses layers support `cudnn`.

## Comments on the code base

This is a simplified version of the original code base we used for all the
experiments in the paper. The complex Torch based class hierarchies in
the `net` reflects varies model architectures we tried during the
experiments. For simplicity, we only make the evaluation part public.
The model detail can be found both in the paper and the model file.

## License

This project is licensed under the terms of the MIT license.


================================================
FILE: cite.bib
================================================
@inproceedings{wang2016interacting,
  title={Interacting with soli: Exploring fine-grained dynamic gesture recognition in the radio-frequency spectrum},
  author={Wang, Saiwen and Song, Jie and Lien, Jaime and Poupyrev, Ivan and Hilliges, Otmar},
  booktitle={Proceedings of the 29th Annual Symposium on User Interface Software and Technology},
  pages={851--860},
  year={2016},
  organization={ACM}
}


================================================
FILE: config/file_half.json
================================================
{"train": ["0_8_4", "0_3_1", "0_2_0", "0_13_5", "0_2_19", "0_3_2", "0_12_1", "0_9_7", "0_3_22", "0_2_2", "0_2_3", "0_5_8", "0_8_21", "0_9_4", "0_2_17", "0_13_13", "0_2_22", "0_9_15", "0_9_17", "0_11_14", "0_10_14", "0_2_16", "0_6_7", "0_2_10", "0_12_19", "0_8_24", "0_8_3", "0_11_4", "0_2_11", "0_3_3", "0_12_14", "0_5_7", "0_11_3", "0_10_1", "0_9_14", "0_11_23", "0_12_13", "0_8_19", "0_8_16", "0_10_16", "0_9_2", "0_8_13", "0_3_9", "0_9_12", "0_13_12", "0_5_15", "0_6_15", "0_6_0", "0_5_0", "0_10_7", "0_9_3", "0_3_11", "0_3_6", "0_13_18", "0_6_19", "0_9_5", "0_10_11", "0_12_7", "0_12_16", "0_11_16", "0_8_14", "0_11_20", "0_10_0", "0_10_13", "0_6_17", "0_6_11", "0_11_17", "0_10_2", "0_11_2", "0_12_18", "0_12_4", "0_11_9", "0_2_14", "0_6_6", "0_5_9", "0_10_8", "0_8_23", "0_11_0", "0_3_20", "0_11_7", "0_11_13", "0_2_7", "0_11_5", "0_8_12", "0_10_21", "0_2_9", "0_12_5", "0_3_23", "0_3_19", "0_8_7", "0_10_10", "0_5_2", "0_13_22", "0_8_5", "0_11_22", "0_11_24", "0_9_24", "0_3_12", "0_2_6", "0_8_20", "0_5_18", "0_5_3", "0_3_8", "0_9_9", "0_11_19", "0_13_23", "0_6_5", "0_11_18", "0_3_13", "0_3_16", "0_9_1", "0_3_4", "0_5_6", "0_6_22", "0_13_20", "0_9_10", "0_3_17", "0_6_10", "0_2_24", "0_10_3", "0_5_5", "0_6_18", "0_6_4", "0_2_1", "0_3_14", "0_10_5", "1_8_3", "1_2_24", "1_2_16", "1_9_12", "1_6_14", "1_13_24", "1_3_6", "1_3_1", "1_9_7", "1_3_24", "1_5_14", "1_9_6", "1_13_6", "1_13_3", "1_2_6", "1_6_10", "1_10_10", "1_2_12", "1_3_4", "1_5_12", "1_12_7", "1_12_24", "1_5_21", "1_12_12", "1_11_9", "1_11_10", "1_13_10", "1_11_19", "1_6_4", "1_5_1", "1_3_16", "1_5_6", "1_6_15", "1_5_18", "1_2_13", "1_9_21", "1_13_19", "1_3_0", "1_13_1", "1_10_12", "1_12_5", "1_9_13", "1_11_22", "1_3_10", "1_3_14", "1_8_11", "1_5_15", "1_6_2", "1_5_9", "1_9_24", "1_8_10", "1_5_10", "1_6_0", "1_10_8", "1_10_2", "1_8_19", "1_13_5", "1_12_11", "1_12_22", "1_9_17", "1_2_18", "1_11_4", "1_2_0", "1_5_22", "1_3_15", "1_10_20", "1_2_5", "1_2_8", "1_13_2", "1_10_5", "1_11_3", "1_10_24", "1_10_21", "1_6_11", "1_11_14", "1_9_1", "1_8_22", "1_13_21", "1_10_14", "1_9_19", "1_6_16", "1_2_10", "1_3_21", "1_8_14", "1_10_1", "1_12_14", "1_2_17", "1_5_19", "1_13_9", "1_8_12", "1_2_11", "1_13_11", "1_3_17", "1_11_17", "1_12_2", "1_12_15", "1_3_9", "1_10_6", "1_11_15", "1_2_23", "1_2_14", "1_8_16", "1_6_3", "1_3_5", "1_11_24", "1_13_14", "1_13_20", "1_3_2", "1_10_18", "1_9_16", "1_9_23", "1_8_2", "1_12_18", "1_3_8", "1_13_7", "1_2_4", "1_2_22", "1_5_8", "1_6_24", "1_2_9", "1_2_2", "1_11_12", "1_11_21", "1_11_8", "1_8_5", "1_11_18", "2_6_6", "2_9_20", "2_2_7", "2_3_16", "2_8_24", "2_8_9", "2_8_21", "2_11_17", "2_8_23", "2_9_8", "2_3_8", "2_2_11", "2_8_18", "2_13_11", "2_9_5", "2_11_14", "2_13_9", "2_12_14", "2_8_16", "2_10_21", "2_2_13", "2_13_18", "2_2_23", "2_12_15", "2_8_12", "2_11_0", "2_5_5", "2_5_19", "2_8_13", "2_5_12", "2_13_21", "2_3_13", "2_9_22", "2_2_9", "2_10_20", "2_6_2", "2_5_3", "2_10_17", "2_11_24", "2_10_12", "2_11_16", "2_11_5", "2_3_0", "2_12_19", "2_3_2", "2_10_0", "2_10_18", "2_13_2", "2_12_17", "2_12_11", "2_13_10", "2_5_17", "2_13_7", "2_11_8", "2_5_6", "2_13_20", "2_8_7", "2_6_7", "2_12_22", "2_6_16", "2_5_21", "2_3_5", "2_5_11", "2_13_17", "2_3_1", "2_6_12", "2_10_24", "2_3_3", "2_8_2", "2_10_23", "2_3_7", "2_5_1", "2_8_11", "2_5_15", "2_6_23", "2_2_5", "2_12_23", "2_5_23", "2_10_22", "2_2_3", "2_13_1", "2_9_7", "2_12_10", "2_12_9", "2_11_6", "2_2_18", "2_12_2", "2_10_10", "2_2_2", "2_11_2", "2_3_20", "2_13_6", "2_12_5", "2_11_18", "2_6_14", "2_5_20", "2_5_8", "2_8_3", "2_10_5", "2_8_4", "2_9_17", "2_2_14", "2_10_19", "2_2_22", "2_2_12", "2_11_21", "2_12_7", "2_6_5", "2_6_13", "2_12_24", "2_5_14", "2_6_8", "2_8_0", "2_8_1", "2_10_7", "2_12_3", "2_3_22", "2_9_24", "2_3_23", "2_12_20", "2_10_11", "2_3_19", "2_5_13", "2_11_9", "2_9_23", "2_2_20", "3_5_21", "3_8_14", "3_9_1", "3_10_21", "3_9_14", "3_12_1", "3_8_24", "3_3_4", "3_12_3", "3_9_24", "3_13_4", "3_2_2", "3_12_14", "3_8_4", "3_6_22", "3_10_16", "3_10_2", "3_5_4", "3_3_13", "3_2_21", "3_11_0", "3_6_16", "3_3_2", "3_8_12", "3_12_18", "3_5_2", "3_6_12", "3_11_13", "3_10_6", "3_6_8", "3_6_21", "3_10_19", "3_11_2", "3_3_5", "3_12_19", "3_13_18", "3_12_12", "3_13_22", "3_3_22", "3_10_8", "3_6_17", "3_11_10", "3_2_10", "3_5_14", "3_6_18", "3_2_9", "3_5_22", "3_9_3", "3_3_8", "3_12_20", "3_2_12", "3_10_1", "3_13_9", "3_13_12", "3_11_20", "3_12_10", "3_6_24", "3_13_2", "3_5_6", "3_12_21", "3_10_17", "3_10_18", "3_9_23", "3_3_12", "3_12_11", "3_8_17", "3_11_6", "3_8_8", "3_11_7", "3_3_19", "3_10_22", "3_10_14", "3_10_20", "3_13_15", "3_5_8", "3_3_10", "3_8_20", "3_11_9", "3_9_13", "3_6_11", "3_6_19", "3_11_18", "3_13_5", "3_8_9", "3_13_21", "3_9_21", "3_2_3", "3_9_7", "3_2_11", "3_3_20", "3_5_15", "3_12_7", "3_5_5", "3_3_17", "3_10_13", "3_11_19", "3_9_4", "3_3_14", "3_10_7", "3_9_11", "3_10_11", "3_10_10", "3_13_17", "3_5_19", "3_2_23", "3_6_5", "3_3_11", "3_13_14", "3_8_10", "3_5_11", "3_9_22", "3_13_13", "3_3_16", "3_11_5", "3_3_18", "3_6_23", "3_5_17", "3_13_1", "3_9_20", "3_5_24", "3_12_22", "3_6_6", "3_12_16", "3_12_15", "3_10_3", "3_5_16", "4_3_14", "4_3_7", "4_3_20", "4_3_5", "4_5_8", "4_2_15", "4_6_10", "4_13_9", "4_13_23", "4_10_16", "4_5_16", "4_10_3", "4_3_1", "4_10_24", "4_2_13", "4_5_11", "4_6_23", "4_9_1", "4_10_23", "4_6_20", "4_2_22", "4_11_18", "4_5_15", "4_9_22", "4_10_20", "4_11_7", "4_12_4", "4_3_0", "4_9_15", "4_9_17", "4_9_5", "4_9_21", "4_9_4", "4_13_24", "4_9_24", "4_10_11", "4_3_9", "4_6_2", "4_9_6", "4_3_3", "4_8_15", "4_3_18", "4_6_14", "4_13_11", "4_8_19", "4_5_4", "4_3_16", "4_6_6", "4_5_7", "4_8_14", "4_11_12", "4_12_14", "4_11_22", "4_11_17", "4_8_24", "4_6_1", "4_13_20", "4_12_21", "4_8_7", "4_3_22", "4_12_17", "4_10_9", "4_11_19", "4_3_17", "4_2_16", "4_10_19", "4_13_21", "4_9_20", "4_8_10", "4_5_2", "4_9_3", "4_9_16", "4_2_7", "4_13_3", "4_10_0", "4_8_22", "4_11_23", "4_5_0", "4_6_0", "4_11_6", "4_9_10", "4_2_6", "4_9_18", "4_8_23", "4_5_19", "4_2_14", "4_12_19", "4_11_4", "4_8_13", "4_8_2", "4_5_18", "4_13_15", "4_13_19", "4_8_5", "4_11_3", "4_3_19", "4_2_23", "4_2_8", "4_3_12", "4_6_18", "4_12_9", "4_2_4", "4_11_0", "4_12_5", "4_12_23", "4_6_15", "4_11_5", "4_3_13", "4_5_24", "4_12_18", "4_13_7", "4_5_12", "4_8_9", "4_6_8", "4_2_3", "4_10_17", "4_5_1", "4_6_24", "4_9_14", "4_3_8", "4_9_19", "4_10_18", "4_5_14", "4_10_1", "4_9_23", "4_12_13", "5_10_0", "5_3_24", "5_12_1", "5_12_4", "5_2_10", "5_8_22", "5_2_21", "5_9_7", "5_6_21", "5_12_14", "5_13_3", "5_12_5", "5_11_8", "5_10_11", "5_9_20", "5_11_1", "5_8_1", "5_12_9", "5_12_22", "5_5_17", "5_6_15", "5_9_23", "5_8_16", "5_6_19", "5_2_11", "5_13_4", "5_11_5", "5_3_20", "5_13_0", "5_8_13", "5_6_5", "5_2_19", "5_9_3", "5_5_10", "5_5_0", "5_9_16", "5_6_0", "5_2_15", "5_5_12", "5_2_4", "5_2_7", "5_3_2", "5_3_11", "5_11_11", "5_3_8", "5_11_13", "5_13_21", "5_5_15", "5_12_24", "5_3_7", "5_3_3", "5_5_21", "5_13_13", "5_10_9", "5_13_6", "5_9_17", "5_5_9", "5_10_20", "5_12_17", "5_11_18", "5_6_16", "5_9_11", "5_2_18", "5_2_9", "5_8_18", "5_13_18", "5_9_5", "5_3_18", "5_2_13", "5_12_3", "5_10_16", "5_3_13", "5_2_17", "5_10_8", "5_5_8", "5_13_2", "5_13_14", "5_13_5", "5_12_6", "5_6_1", "5_11_4", "5_3_5", "5_12_0", "5_6_2", "5_12_18", "5_6_13", "5_13_1", "5_9_13", "5_10_24", "5_3_6", "5_5_23", "5_12_15", "5_12_7", "5_13_9", "5_2_6", "5_10_14", "5_8_11", "5_10_15", "5_6_3", "5_9_0", "5_9_22", "5_8_10", "5_8_12", "5_8_19", "5_2_3", "5_2_2", "5_13_8", "5_10_2", "5_9_6", "5_8_4", "5_10_6", "5_6_14", "5_9_10", "5_3_21", "5_11_12", "5_8_5", "5_10_22", "5_13_12", "5_8_23", "5_5_4", "5_10_13", "5_5_19", "5_8_8", "5_6_17", "5_3_17", "5_5_11", "6_11_2", "6_11_21", "6_3_5", "6_6_19", "6_12_4", "6_3_24", "6_13_0", "6_6_16", "6_3_18", "6_3_13", "6_9_20", "6_9_24", "6_9_21", "6_2_12", "6_13_20", "6_6_22", "6_9_13", "6_8_3", "6_2_20", "6_2_4", "6_10_0", "6_9_6", "6_10_13", "6_3_11", "6_11_11", "6_8_13", "6_8_8", "6_9_9", "6_12_10", "6_10_7", "6_12_16", "6_3_14", "6_8_17", "6_6_11", "6_9_2", "6_8_0", "6_10_1", "6_5_5", "6_3_4", "6_2_16", "6_8_2", "6_6_2", "6_10_8", "6_9_3", "6_9_14", "6_10_12", "6_2_1", "6_9_0", "6_2_15", "6_9_15", "6_10_14", "6_5_13", "6_3_9", "6_12_24", "6_3_2", "6_6_10", "6_5_20", "6_9_22", "6_6_9", "6_9_1", "6_6_24", "6_13_7", "6_5_2", "6_10_20", "6_12_9", "6_10_3", "6_9_8", "6_2_8", "6_12_23", "6_10_5", "6_8_15", "6_13_12", "6_9_16", "6_10_10", "6_5_14", "6_13_1", "6_11_3", "6_13_13", "6_3_20", "6_9_11", "6_5_12", "6_12_11", "6_9_18", "6_6_21", "6_8_12", "6_5_10", "6_8_23", "6_3_6", "6_9_12", "6_8_9", "6_5_1", "6_6_20", "6_11_19", "6_10_19", "6_8_20", "6_5_11", "6_5_8", "6_9_4", "6_2_21", "6_11_1", "6_6_13", "6_10_15", "6_2_7", "6_8_14", "6_3_3", "6_9_10", "6_5_0", "6_6_5", "6_3_0", "6_10_4", "6_6_23", "6_12_12", "6_3_15", "6_8_11", "6_10_21", "6_3_10", "6_13_5", "6_6_15", "6_12_6", "6_10_9", "6_11_23", "6_5_4", "6_6_7", "6_8_21", "6_12_21", "6_8_18", "7_12_6", "7_8_1", "7_11_17", "7_2_3", "7_10_14", "7_6_9", "7_5_9", "7_6_23", "7_5_14", "7_3_13", "7_2_5", "7_13_20", "7_10_24", "7_13_21", "7_5_20", "7_8_20", "7_12_18", "7_8_18", "7_11_16", "7_3_17", "7_10_21", "7_5_13", "7_2_13", "7_11_3", "7_5_4", "7_2_20", "7_9_10", "7_6_24", "7_13_15", "7_2_21", "7_2_4", "7_8_7", "7_13_9", "7_8_4", "7_5_5", "7_13_8", "7_6_2", "7_13_10", "7_9_13", "7_8_23", "7_9_11", "7_8_11", "7_9_19", "7_3_12", "7_10_6", "7_8_10", "7_3_3", "7_11_15", "7_3_7", "7_6_14", "7_2_9", "7_11_13", "7_12_7", "7_3_10", "7_5_8", "7_2_24", "7_10_12", "7_11_7", "7_11_10", "7_13_16", "7_10_1", "7_6_15", "7_8_5", "7_3_20", "7_6_0", "7_9_8", "7_12_17", "7_12_24", "7_2_0", "7_10_20", "7_10_2", "7_9_1", "7_11_5", "7_12_16", "7_11_12", "7_9_9", "7_3_2", "7_6_21", "7_13_5", "7_10_23", "7_11_14", "7_9_3", "7_2_15", "7_6_7", "7_2_17", "7_6_3", "7_5_22", "7_9_21", "7_12_23", "7_10_18", "7_3_14", "7_8_2", "7_5_23", "7_12_5", "7_13_23", "7_2_16", "7_12_22", "7_9_4", "7_13_17", "7_6_4", "7_10_11", "7_10_7", "7_3_1", "7_5_15", "7_9_14", "7_2_1", "7_9_12", "7_5_11", "7_11_9", "7_6_20", "7_6_19", "7_12_0", "7_13_2", "7_6_1", "7_13_11", "7_11_21", "7_8_19", "7_11_23", "7_8_13", "7_12_20", "7_2_22", "7_11_22", "7_10_22", "7_9_0", "7_3_19", "7_3_5", "8_11_7", "8_3_20", "8_13_6", "8_2_5", "8_13_16", "8_13_2", "8_10_14", "8_13_20", "8_8_0", "8_2_20", "8_2_24", "8_2_7", "8_11_2", "8_2_12", "8_2_14", "8_2_17", "8_10_11", "8_6_22", "8_10_19", "8_12_17", "8_2_11", "8_9_9", "8_9_0", "8_11_11", "8_13_0", "8_11_6", "8_5_24", "8_10_5", "8_5_9", "8_13_4", "8_3_2", "8_6_16", "8_13_5", "8_8_11", "8_5_19", "8_10_7", "8_12_13", "8_13_14", "8_8_3", "8_12_18", "8_3_16", "8_12_23", "8_10_8", "8_11_17", "8_8_16", "8_3_9", "8_8_22", "8_11_3", "8_10_24", "8_3_23", "8_6_13", "8_5_17", "8_2_4", "8_13_21", "8_12_1", "8_10_17", "8_6_2", "8_6_7", "8_11_24", "8_3_10", "8_12_20", "8_3_3", "8_12_14", "8_5_6", "8_8_18", "8_11_13", "8_2_23", "8_2_1", "8_12_12", "8_8_6", "8_12_16", "8_5_4", "8_3_6", "8_12_2", "8_10_3", "8_5_23", "8_12_4", "8_11_12", "8_6_5", "8_13_24", "8_3_19", "8_10_18", "8_12_21", "8_9_16", "8_9_3", "8_3_22", "8_8_9", "8_3_17", "8_9_7", "8_5_22", "8_8_10", "8_9_11", "8_12_3", "8_9_22", "8_2_13", "8_6_23", "8_6_20", "8_6_18", "8_12_7", "8_10_0", "8_8_7", "8_12_19", "8_8_20", "8_12_9", "8_9_14", "8_12_15", "8_3_4", "8_11_1", "8_6_14", "8_6_15", "8_9_13", "8_10_10", "8_3_12", "8_5_12", "8_2_21", "8_13_7", "8_10_20", "8_6_3", "8_8_8", "8_6_12", "8_5_3", "8_6_8", "8_11_0", "8_8_19", "8_11_19", "8_9_5", "9_8_3", "9_9_18", "9_8_13", "9_9_16", "9_8_5", "9_11_23", "9_6_1", "9_9_17", "9_3_20", "9_2_23", "9_9_10", "9_12_21", "9_5_21", "9_10_10", "9_8_22", "9_6_7", "9_3_23", "9_9_21", "9_11_8", "9_2_17", "9_9_12", "9_11_15", "9_11_6", "9_2_21", "9_2_0", "9_13_1", "9_9_20", "9_5_22", "9_12_20", "9_2_14", "9_10_23", "9_5_7", "9_12_0", "9_3_11", "9_10_5", "9_2_16", "9_5_12", "9_8_16", "9_13_9", "9_12_15", "9_12_24", "9_5_14", "9_9_13", "9_13_0", "9_6_4", "9_9_23", "9_13_19", "9_13_12", "9_9_4", "9_9_5", "9_8_12", "9_13_23", "9_5_9", "9_8_14", "9_13_4", "9_3_10", "9_5_11", "9_3_9", "9_3_3", "9_11_14", "9_2_22", "9_9_15", "9_13_3", "9_10_19", "9_11_12", "9_12_22", "9_8_1", "9_11_5", "9_10_16", "9_3_21", "9_10_18", "9_12_17", "9_10_15", "9_3_7", "9_5_18", "9_5_19", "9_2_4", "9_12_3", "9_3_17", "9_2_5", "9_13_24", "9_6_17", "9_6_11", "9_11_21", "9_10_0", "9_11_18", "9_10_11", "9_8_24", "9_12_18", "9_10_2", "9_2_8", "9_9_3", "9_11_9", "9_5_23", "9_5_15", "9_12_6", "9_9_22", "9_2_10", "9_2_3", "9_10_12", "9_11_17", "9_3_14", "9_13_14", "9_8_0", "9_3_8", "9_5_1", "9_13_17", "9_12_12", "9_6_19", "9_3_15", "9_8_11", "9_3_4", "9_10_21", "9_6_21", "9_10_9", "9_8_2", "9_6_10", "9_8_4", "9_2_13", "9_3_1", "9_6_20", "9_10_14", "9_8_7", "9_3_13", "9_5_0", "9_5_17", "10_10_16", "10_10_7", "10_3_23", "10_5_0", "10_6_16", "10_10_11", "10_6_2", "10_5_9", "10_12_7", "10_10_19", "10_13_15", "10_10_3", "10_6_7", "10_11_17", "10_9_0", "10_6_5", "10_3_11", "10_13_0", "10_13_2", "10_10_13", "10_11_20", "10_3_19", "10_13_14", "10_5_15", "10_9_19", "10_9_22", "10_11_3", "10_2_1", "10_6_13", "10_12_11", "10_9_20", "10_8_6", "10_10_20", "10_8_2", "10_8_19", "10_9_14", "10_2_14", "10_13_18", "10_9_16", "10_2_13", "10_9_7", "10_3_1", "10_9_4", "10_2_15", "10_9_6", "10_8_4", "10_12_13", "10_13_10", "10_12_1", "10_12_2", "10_13_13", "10_8_0", "10_10_6", "10_12_20", "10_5_16", "10_2_20", "10_6_8", "10_12_15", "10_2_2", "10_9_1", "10_13_19", "10_9_15", "10_8_20", "10_8_22", "10_2_8", "10_2_7", "10_11_23", "10_10_1", "10_9_17", "10_10_5", "10_10_18", "10_11_8", "10_6_17", "10_11_12", "10_3_15", "10_13_20", "10_3_17", "10_11_0", "10_2_24", "10_12_12", "10_12_10", "10_13_4", "10_8_16", "10_8_11", "10_8_3", "10_6_18", "10_8_10", "10_5_14", "10_12_17", "10_6_23", "10_3_13", "10_13_12", "10_5_19", "10_6_20", "10_11_21", "10_11_7", "10_12_4", "10_13_5", "10_2_12", "10_11_6", "10_10_15", "10_2_19", "10_10_22", "10_13_9", "10_5_5", "10_5_1", "10_5_24", "10_2_9", "10_6_22", "10_12_16", "10_5_6", "10_13_11", "10_13_21", "10_10_0", "10_9_10", "10_10_8", "10_9_3", "10_5_12", "10_12_19", "10_8_9", "10_3_12", "10_11_11", "10_11_16", "10_5_4", "10_6_19", "10_3_7"], "eval": ["0_12_20", "0_10_18", "0_6_8", "0_3_0", "0_12_22", "0_13_10", "0_12_3", "0_5_14", "0_10_15", "0_2_5", "0_3_5", "0_3_18", "0_12_21", "0_5_21", "0_10_20", "0_8_22", "0_8_11", "0_13_6", "0_9_0", "0_3_7", "0_8_8", "0_5_10", "0_10_6", "0_9_20", "0_6_20", "0_13_15", "0_6_2", "0_9_13", "0_13_17", "0_12_23", "0_2_4", "0_12_10", "0_11_12", "0_13_21", "0_13_9", "0_12_6", "0_5_12", "0_9_16", "0_2_18", "0_13_4", "0_6_1", "0_10_12", "0_11_1", "0_11_6", "0_13_16", "0_2_13", "0_2_12", "0_5_16", "0_8_10", "0_13_19", "0_12_2", "0_12_17", "0_9_19", "0_6_3", "0_13_2", "0_9_6", "0_5_13", "0_12_9", "0_6_21", "0_5_19", "0_10_9", "0_9_23", "0_2_15", "0_9_22", "0_3_15", "0_11_21", "0_8_18", "0_12_12", "0_12_8", "0_5_22", "0_10_23", "0_5_17", "0_13_24", "0_6_12", "0_6_13", "0_10_24", "0_8_0", "0_5_24", "0_12_0", "0_8_6", "0_8_15", "0_5_11", "0_2_20", "0_8_2", "0_6_16", "0_9_21", "0_6_14", "0_3_21", "0_10_4", "0_13_8", "0_10_22", "0_9_18", "0_6_24", "0_13_1", "0_11_10", "0_5_4", "0_8_1", "0_13_11", "0_10_19", "0_2_23", "0_13_3", "0_11_8", "0_8_9", "0_12_15", "0_11_15", "0_3_24", "0_8_17", "0_3_10", "0_5_1", "0_2_21", "0_10_17", "0_2_8", "0_9_8", "0_13_7", "0_6_9", "0_5_20", "0_12_24", "0_12_11", "0_13_0", "0_9_11", "0_5_23", "0_11_11", "0_6_23", "0_13_14", "1_6_23", "1_9_5", "1_11_16", "1_10_16", "1_13_22", "1_3_18", "1_6_19", "1_8_23", "1_9_20", "1_8_9", "1_8_24", "1_9_15", "1_12_9", "1_5_2", "1_5_24", "1_6_17", "1_10_3", "1_10_22", "1_6_7", "1_12_1", "1_9_10", "1_12_10", "1_6_5", "1_8_21", "1_5_17", "1_3_20", "1_13_23", "1_2_15", "1_8_7", "1_11_1", "1_2_7", "1_11_23", "1_13_8", "1_6_13", "1_2_21", "1_3_13", "1_13_15", "1_6_18", "1_9_9", "1_2_19", "1_12_17", "1_11_11", "1_5_20", "1_5_16", "1_12_6", "1_3_7", "1_2_20", "1_11_7", "1_6_1", "1_9_22", "1_10_0", "1_6_22", "1_9_14", "1_2_3", "1_6_6", "1_8_17", "1_3_19", "1_11_0", "1_8_18", "1_10_15", "1_9_8", "1_9_18", "1_5_0", "1_10_23", "1_5_23", "1_10_13", "1_10_7", "1_6_8", "1_11_6", "1_8_0", "1_8_6", "1_13_17", "1_12_4", "1_13_13", "1_9_0", "1_6_20", "1_5_4", "1_3_3", "1_3_23", "1_12_20", "1_10_4", "1_11_2", "1_8_20", "1_12_3", "1_8_8", "1_12_8", "1_12_0", "1_5_11", "1_13_12", "1_9_11", "1_2_1", "1_12_13", "1_10_17", "1_3_22", "1_6_9", "1_3_12", "1_12_16", "1_8_15", "1_12_23", "1_5_5", "1_11_5", "1_11_20", "1_13_16", "1_8_4", "1_5_13", "1_13_18", "1_8_13", "1_8_1", "1_10_9", "1_13_4", "1_6_12", "1_5_7", "1_6_21", "1_9_2", "1_3_11", "1_5_3", "1_10_19", "1_9_4", "1_9_3", "1_13_0", "1_12_19", "1_10_11", "1_11_13", "1_12_21", "2_6_24", "2_10_13", "2_12_18", "2_5_2", "2_2_10", "2_12_4", "2_2_15", "2_10_14", "2_9_11", "2_9_19", "2_8_5", "2_9_14", "2_10_6", "2_3_18", "2_13_12", "2_2_0", "2_11_19", "2_12_21", "2_5_0", "2_8_19", "2_9_18", "2_13_5", "2_3_12", "2_3_6", "2_12_6", "2_9_21", "2_9_4", "2_3_14", "2_11_10", "2_11_1", "2_9_6", "2_2_17", "2_5_16", "2_6_9", "2_2_16", "2_6_11", "2_9_9", "2_9_12", "2_8_15", "2_2_1", "2_6_10", "2_13_16", "2_12_1", "2_9_13", "2_10_3", "2_6_4", "2_6_1", "2_11_12", "2_12_12", "2_11_7", "2_3_17", "2_6_17", "2_9_0", "2_2_8", "2_3_24", "2_9_3", "2_8_6", "2_13_0", "2_9_2", "2_5_10", "2_13_4", "2_3_10", "2_6_3", "2_9_1", "2_3_11", "2_9_10", "2_13_14", "2_5_7", "2_6_19", "2_13_19", "2_10_1", "2_2_19", "2_11_20", "2_13_24", "2_10_4", "2_6_18", "2_9_15", "2_6_15", "2_5_22", "2_13_3", "2_10_16", "2_11_11", "2_8_17", "2_10_2", "2_6_0", "2_3_21", "2_3_15", "2_13_8", "2_6_22", "2_10_9", "2_12_16", "2_5_18", "2_8_10", "2_10_15", "2_8_22", "2_13_13", "2_12_8", "2_13_15", "2_11_22", "2_2_4", "2_11_13", "2_3_4", "2_5_4", "2_11_4", "2_8_14", "2_2_6", "2_9_16", "2_11_23", "2_12_13", "2_3_9", "2_8_20", "2_6_20", "2_11_15", "2_6_21", "2_13_22", "2_5_9", "2_10_8", "2_12_0", "2_11_3", "2_2_24", "2_5_24", "2_13_23", "2_8_8", "2_2_21", "3_13_24", "3_2_17", "3_11_3", "3_5_9", "3_2_0", "3_3_23", "3_10_23", "3_3_6", "3_10_9", "3_8_18", "3_5_12", "3_11_17", "3_8_5", "3_2_15", "3_6_14", "3_9_12", "3_9_9", "3_13_10", "3_11_14", "3_8_3", "3_2_8", "3_11_8", "3_8_1", "3_3_15", "3_5_0", "3_13_0", "3_8_2", "3_6_7", "3_13_3", "3_11_1", "3_2_20", "3_8_16", "3_5_3", "3_2_13", "3_2_7", "3_2_4", "3_5_18", "3_6_3", "3_3_7", "3_9_18", "3_6_15", "3_9_15", "3_2_24", "3_11_22", "3_2_18", "3_10_12", "3_8_0", "3_8_13", "3_8_21", "3_9_6", "3_3_21", "3_12_13", "3_3_1", "3_8_15", "3_11_4", "3_3_0", "3_13_19", "3_11_16", "3_6_9", "3_2_19", "3_9_17", "3_13_11", "3_5_20", "3_13_7", "3_3_3", "3_13_23", "3_9_16", "3_9_8", "3_5_13", "3_5_23", "3_10_15", "3_8_19", "3_12_5", "3_12_4", "3_9_19", "3_6_20", "3_11_21", "3_6_1", "3_11_12", "3_2_22", "3_2_5", "3_6_13", "3_12_6", "3_2_16", "3_9_0", "3_8_23", "3_9_10", "3_6_0", "3_11_15", "3_12_9", "3_5_1", "3_13_20", "3_13_8", "3_13_6", "3_10_5", "3_10_24", "3_8_22", "3_6_10", "3_8_6", "3_8_7", "3_9_5", "3_11_11", "3_2_14", "3_6_2", "3_5_10", "3_11_23", "3_8_11", "3_10_0", "3_2_1", "3_11_24", "3_12_23", "3_12_17", "3_12_0", "3_12_24", "3_12_2", "3_9_2", "3_3_9", "3_5_7", "3_13_16", "3_10_4", "3_12_8", "3_6_4", "3_2_6", "3_3_24", "4_12_3", "4_6_12", "4_10_13", "4_3_21", "4_6_21", "4_3_15", "4_11_16", "4_13_5", "4_3_4", "4_10_2", "4_6_7", "4_9_11", "4_10_8", "4_2_20", "4_3_6", "4_3_2", "4_11_11", "4_8_16", "4_2_0", "4_2_2", "4_5_6", "4_12_1", "4_6_19", "4_8_0", "4_12_8", "4_13_10", "4_2_21", "4_3_24", "4_12_22", "4_11_8", "4_5_17", "4_13_4", "4_8_1", "4_11_14", "4_11_20", "4_10_21", "4_13_2", "4_13_1", "4_8_12", "4_9_0", "4_12_11", "4_12_2", "4_2_24", "4_3_11", "4_6_16", "4_12_7", "4_6_3", "4_9_13", "4_9_2", "4_13_6", "4_6_9", "4_8_18", "4_12_24", "4_2_11", "4_2_18", "4_9_8", "4_5_22", "4_5_10", "4_11_24", "4_13_22", "4_13_17", "4_8_11", "4_13_14", "4_8_20", "4_11_13", "4_2_1", "4_12_16", "4_11_15", "4_13_0", "4_2_5", "4_12_20", "4_5_5", "4_10_12", "4_8_21", "4_13_13", "4_10_6", "4_9_7", "4_2_19", "4_10_15", "4_10_22", "4_11_1", "4_10_7", "4_12_0", "4_2_12", "4_12_15", "4_9_9", "4_8_17", "4_8_8", "4_12_10", "4_3_23", "4_6_22", "4_8_4", "4_11_10", "4_2_9", "4_5_23", "4_6_4", "4_5_3", "4_13_12", "4_3_10", "4_10_4", "4_12_6", "4_6_13", "4_10_10", "4_9_12", "4_8_6", "4_13_16", "4_2_17", "4_2_10", "4_13_8", "4_11_21", "4_12_12", "4_11_9", "4_8_3", "4_13_18", "4_5_9", "4_5_20", "4_10_14", "4_6_5", "4_10_5", "4_11_2", "4_6_11", "4_5_13", "4_5_21", "4_6_17", "5_8_21", "5_10_19", "5_2_8", "5_5_16", "5_9_21", "5_2_20", "5_6_18", "5_6_8", "5_9_18", "5_12_11", "5_10_17", "5_13_11", "5_8_3", "5_5_13", "5_11_7", "5_12_8", "5_2_0", "5_3_23", "5_8_17", "5_5_6", "5_10_5", "5_8_9", "5_10_21", "5_12_10", "5_3_16", "5_5_14", "5_9_1", "5_11_21", "5_11_16", "5_5_22", "5_2_24", "5_13_15", "5_8_0", "5_10_1", "5_8_14", "5_8_20", "5_12_19", "5_8_6", "5_6_7", "5_5_1", "5_11_14", "5_3_19", "5_11_9", "5_5_24", "5_6_20", "5_8_24", "5_3_12", "5_12_2", "5_5_2", "5_12_16", "5_3_0", "5_11_10", "5_11_20", "5_13_19", "5_5_5", "5_3_15", "5_9_14", "5_9_24", "5_13_24", "5_11_23", "5_13_7", "5_13_23", "5_6_9", "5_6_24", "5_2_12", "5_5_7", "5_3_22", "5_12_12", "5_5_18", "5_12_23", "5_13_22", "5_2_23", "5_11_24", "5_11_17", "5_2_14", "5_3_4", "5_11_6", "5_11_0", "5_6_6", "5_10_4", "5_5_20", "5_11_15", "5_10_3", "5_3_10", "5_11_2", "5_9_4", "5_9_2", "5_3_14", "5_10_7", "5_10_23", "5_11_3", "5_6_4", "5_9_19", "5_9_8", "5_6_22", "5_8_7", "5_6_11", "5_9_9", "5_6_23", "5_9_15", "5_3_1", "5_12_20", "5_5_3", "5_12_21", "5_2_22", "5_13_16", "5_8_15", "5_10_18", "5_13_20", "5_9_12", "5_6_10", "5_10_10", "5_13_10", "5_8_2", "5_13_17", "5_10_12", "5_12_13", "5_3_9", "5_6_12", "5_2_16", "5_11_19", "5_2_5", "5_2_1", "5_11_22", "6_2_11", "6_2_17", "6_11_13", "6_2_24", "6_13_18", "6_5_19", "6_6_14", "6_12_1", "6_13_23", "6_5_15", "6_10_6", "6_12_19", "6_12_22", "6_6_12", "6_6_1", "6_10_17", "6_3_23", "6_13_9", "6_9_5", "6_13_19", "6_6_4", "6_5_22", "6_12_15", "6_5_21", "6_8_1", "6_12_7", "6_2_9", "6_11_16", "6_2_5", "6_2_18", "6_5_7", "6_6_6", "6_11_7", "6_11_6", "6_12_14", "6_11_9", "6_12_20", "6_8_16", "6_10_18", "6_2_19", "6_10_2", "6_2_22", "6_3_22", "6_6_18", "6_11_22", "6_13_22", "6_5_24", "6_3_19", "6_12_5", "6_5_17", "6_8_7", "6_8_19", "6_8_6", "6_13_8", "6_13_2", "6_9_19", "6_11_17", "6_11_4", "6_9_7", "6_13_21", "6_5_6", "6_11_0", "6_5_18", "6_2_10", "6_13_3", "6_13_24", "6_8_10", "6_12_0", "6_10_24", "6_13_11", "6_5_23", "6_2_3", "6_13_16", "6_13_6", "6_3_12", "6_12_17", "6_11_15", "6_11_24", "6_12_2", "6_3_21", "6_12_18", "6_9_23", "6_8_24", "6_2_13", "6_6_17", "6_6_0", "6_6_3", "6_6_8", "6_13_15", "6_3_8", "6_11_10", "6_10_16", "6_8_22", "6_2_2", "6_5_9", "6_8_5", "6_2_23", "6_11_20", "6_10_11", "6_11_14", "6_2_6", "6_2_0", "6_3_1", "6_3_17", "6_13_4", "6_13_10", "6_10_22", "6_11_8", "6_5_16", "6_8_4", "6_13_17", "6_2_14", "6_10_23", "6_5_3", "6_11_5", "6_12_8", "6_11_12", "6_13_14", "6_9_17", "6_11_18", "6_12_3", "6_12_13", "6_3_16", "6_3_7", "7_6_5", "7_12_19", "7_11_18", "7_2_6", "7_9_23", "7_3_21", "7_3_23", "7_12_2", "7_12_14", "7_6_16", "7_12_3", "7_8_9", "7_10_4", "7_2_12", "7_12_8", "7_5_12", "7_9_15", "7_9_16", "7_6_11", "7_9_5", "7_11_4", "7_8_0", "7_5_21", "7_5_19", "7_5_10", "7_9_7", "7_2_7", "7_8_22", "7_9_24", "7_10_5", "7_10_15", "7_5_7", "7_10_13", "7_6_17", "7_2_23", "7_9_20", "7_11_20", "7_13_14", "7_12_12", "7_9_2", "7_3_22", "7_3_18", "7_5_3", "7_9_6", "7_11_0", "7_8_15", "7_6_6", "7_5_1", "7_13_24", "7_3_4", "7_9_17", "7_5_2", "7_12_1", "7_13_0", "7_9_22", "7_12_9", "7_5_17", "7_12_10", "7_13_3", "7_2_10", "7_6_8", "7_2_18", "7_6_18", "7_13_13", "7_11_1", "7_12_21", "7_12_15", "7_13_18", "7_8_21", "7_13_7", "7_3_9", "7_3_8", "7_2_11", "7_8_12", "7_2_2", "7_12_4", "7_6_12", "7_10_16", "7_13_4", "7_5_16", "7_10_9", "7_11_19", "7_3_16", "7_3_15", "7_2_8", "7_11_2", "7_3_24", "7_6_13", "7_10_19", "7_10_0", "7_8_6", "7_10_17", "7_10_3", "7_8_14", "7_13_22", "7_2_19", "7_8_8", "7_3_11", "7_3_6", "7_5_24", "7_5_0", "7_3_0", "7_8_17", "7_12_11", "7_5_18", "7_11_6", "7_8_3", "7_6_22", "7_11_11", "7_9_18", "7_5_6", "7_13_19", "7_10_8", "7_11_24", "7_12_13", "7_8_24", "7_13_6", "7_13_12", "7_8_16", "7_10_10", "7_13_1", "7_6_10", "7_11_8", "7_2_14", "8_11_15", "8_12_10", "8_13_11", "8_10_6", "8_6_21", "8_5_10", "8_9_10", "8_13_17", "8_10_22", "8_9_20", "8_8_12", "8_8_2", "8_5_2", "8_11_23", "8_13_22", "8_3_21", "8_12_8", "8_2_16", "8_3_18", "8_8_17", "8_9_15", "8_2_10", "8_5_1", "8_9_23", "8_9_2", "8_8_1", "8_6_10", "8_5_11", "8_2_3", "8_5_7", "8_2_6", "8_11_8", "8_8_4", "8_6_9", "8_5_5", "8_9_8", "8_10_1", "8_13_1", "8_9_1", "8_3_0", "8_3_13", "8_11_21", "8_8_5", "8_13_13", "8_6_24", "8_6_11", "8_2_9", "8_5_15", "8_5_13", "8_2_8", "8_13_15", "8_10_12", "8_11_10", "8_10_2", "8_5_0", "8_9_24", "8_9_21", "8_3_15", "8_2_19", "8_2_22", "8_10_13", "8_5_8", "8_5_16", "8_9_6", "8_9_19", "8_6_17", "8_3_7", "8_11_5", "8_6_19", "8_12_0", "8_2_2", "8_2_0", "8_10_4", "8_8_23", "8_10_16", "8_11_18", "8_13_3", "8_11_9", "8_3_14", "8_3_5", "8_13_8", "8_5_18", "8_10_23", "8_2_18", "8_13_23", "8_9_18", "8_13_12", "8_8_14", "8_12_24", "8_12_5", "8_9_12", "8_3_1", "8_11_16", "8_6_6", "8_12_11", "8_8_15", "8_8_24", "8_6_4", "8_2_15", "8_11_20", "8_11_14", "8_13_19", "8_10_15", "8_10_9", "8_13_10", "8_12_22", "8_13_9", "8_11_22", "8_5_20", "8_11_4", "8_3_24", "8_8_13", "8_9_17", "8_6_0", "8_12_6", "8_3_11", "8_6_1", "8_9_4", "8_13_18", "8_5_14", "8_10_21", "8_8_21", "8_5_21", "8_3_8", "9_2_1", "9_9_14", "9_10_1", "9_9_1", "9_13_2", "9_3_0", "9_11_24", "9_11_0", "9_12_23", "9_2_24", "9_6_8", "9_11_3", "9_6_9", "9_2_18", "9_6_24", "9_11_16", "9_2_12", "9_2_15", "9_2_20", "9_13_13", "9_13_6", "9_6_14", "9_2_9", "9_2_11", "9_6_22", "9_3_16", "9_9_2", "9_9_19", "9_13_16", "9_9_8", "9_12_4", "9_13_7", "9_12_7", "9_13_10", "9_3_19", "9_8_10", "9_10_13", "9_13_15", "9_8_19", "9_12_1", "9_11_4", "9_12_10", "9_13_5", "9_10_3", "9_12_19", "9_3_18", "9_5_16", "9_3_22", "9_11_22", "9_5_8", "9_5_20", "9_13_22", "9_10_17", "9_12_16", "9_10_24", "9_13_8", "9_8_15", "9_13_18", "9_3_24", "9_2_2", "9_8_20", "9_8_17", "9_10_20", "9_11_2", "9_9_6", "9_8_9", "9_8_23", "9_5_10", "9_8_18", "9_12_9", "9_10_7", "9_3_2", "9_13_11", "9_6_3", "9_9_24", "9_6_18", "9_6_0", "9_10_8", "9_9_9", "9_6_6", "9_11_7", "9_6_5", "9_11_13", "9_13_20", "9_11_10", "9_9_11", "9_10_22", "9_11_19", "9_6_16", "9_12_13", "9_6_12", "9_12_5", "9_9_7", "9_5_6", "9_5_5", "9_12_14", "9_9_0", "9_6_23", "9_10_4", "9_3_5", "9_6_2", "9_3_6", "9_6_13", "9_11_11", "9_5_13", "9_2_19", "9_12_11", "9_11_20", "9_8_21", "9_11_1", "9_5_4", "9_2_6", "9_5_3", "9_5_2", "9_8_8", "9_6_15", "9_2_7", "9_5_24", "9_12_8", "9_13_21", "9_3_12", "9_8_6", "9_10_6", "9_12_2", "10_13_24", "10_13_16", "10_12_8", "10_3_2", "10_12_18", "10_12_3", "10_9_2", "10_10_4", "10_9_9", "10_9_5", "10_8_18", "10_2_6", "10_13_1", "10_9_21", "10_11_1", "10_10_17", "10_13_17", "10_2_17", "10_13_6", "10_12_24", "10_2_5", "10_9_11", "10_13_7", "10_9_8", "10_10_2", "10_5_8", "10_12_23", "10_6_11", "10_3_24", "10_5_3", "10_6_15", "10_11_22", "10_9_24", "10_10_23", "10_9_18", "10_8_23", "10_6_6", "10_2_21", "10_10_21", "10_8_12", "10_2_3", "10_8_7", "10_6_9", "10_8_14", "10_5_17", "10_3_16", "10_11_5", "10_8_8", "10_3_21", "10_3_18", "10_3_10", "10_2_10", "10_8_15", "10_8_5", "10_6_4", "10_10_14", "10_2_4", "10_3_0", "10_11_18", "10_9_12", "10_6_24", "10_12_5", "10_13_3", "10_3_8", "10_3_9", "10_12_0", "10_6_10", "10_2_23", "10_9_23", "10_11_10", "10_11_9", "10_11_14", "10_2_16", "10_13_8", "10_3_22", "10_5_13", "10_5_22", "10_12_9", "10_3_6", "10_5_21", "10_5_2", "10_10_9", "10_11_19", "10_5_20", "10_3_14", "10_5_10", "10_11_15", "10_10_10", "10_8_21", "10_12_22", "10_11_24", "10_6_14", "10_12_14", "10_5_7", "10_11_13", "10_8_24", "10_6_12", "10_6_21", "10_6_1", "10_5_23", "10_6_0", "10_5_18", "10_11_4", "10_12_21", "10_3_3", "10_2_11", "10_11_2", "10_5_11", "10_6_3", "10_10_24", "10_9_13", "10_8_13", "10_8_17", "10_3_20", "10_13_22", "10_2_22", "10_13_23", "10_8_1", "10_10_12", "10_2_18", "10_3_4", "10_2_0", "10_12_6", "10_3_5"]}


================================================
FILE: image/Makefile
================================================
CC=c++

# c flags
CFLAGS=-c -Wall -std=c++11 -O3
CFLAGS_BOOST=-DBOOST_SYSTEM_NO_DEPRECATED
CFLAGS_CV=`pkg-config --cflags opencv`
CFLAGS_H5=-I/usr/include -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE \
-D_BSD_SOURCE -D_FORTIFY_SOURCE=2 -g -fstack-protector \
--param=ssp-buffer-size=4 -Wformat -Werror=format-security

# linking flags
LDFLAGS_BOOST=-lboost_system -lboost_program_options
LDFLAGS_H5=-lhdf5 -lhdf5_cpp
LDFLAGS_CV=-L/usr/local/cuda-7.5/lib64 `pkg-config --libs opencv`

BIN=bin
SOURCES=$(wildcard *.cpp)
OBJECTS=$(patsubst %.cpp, %.o, $(SOURCES))
EXECUTABLE=$(BIN)/image

all: $(SOURCES) $(EXECUTABLE)

$(BIN)/image: $(BIN)/main.o $(BIN)/h5Data.o $(BIN)/seq.o
	$(CC) $(BIN)/main.o $(BIN)/h5Data.o $(BIN)/seq.o \
	$(LDFLAGS_BOOST) $(LDFLAGS_H5) $(LDFLAGS_CV) -o $@

$(BIN)/main.o: main.cpp
	$(CC) $(CFLAGS) $(CFLAGS_BOOST) $< -o $@

$(BIN)/h5Data.o: h5Data.cpp
	$(CC) $(CFLAGS) $(CFLAGS_H5) $< -o $@

$(BIN)/seq.o: seq.cpp
	$(CC) $(CFLAGS) $(CFLAGS_CV) $< -o $@

clean:
	rm -r -f $(BIN)/*


================================================
FILE: image/h5Data.cpp
================================================
#include "h5Data.hpp"

h5Data::h5Data(std::string f, int n) : fileName(f), numChannel(n) {
	// turn off the auto-printing when failure occurs
	dataFile = new H5::H5File(fileName, H5F_ACC_RDONLY);

	datasetLa = load(datasetNameLa, -1);
	for (int i = 0; i < numChannel; i++) {
		datasetCh[i] = load(datasetNameCh + std::to_string(i), i);
	}
}

h5Data::~h5Data() {
	// delete array of label and channels with []
	delete[] label;
	delete datasetLa;
	for (int i = 0; i < numChannel; i++) {
		delete[] ch[i];
		delete datasetCh[i];
	}

	delete dataFile;
}

H5::DataSet* h5Data::load(H5std_string datasetName, int whichCh) {
	// open dataset, dataspace and property list
	try {
		H5::Exception::dontPrint();
		hsize_t dimsr[2];
		H5::DataSet* dataset = new H5::DataSet(dataFile->openDataSet(datasetName));
		H5::DataSpace *filespace = new H5::DataSpace(dataset->getSpace());
		H5::DSetCreatPropList prop = dataset->getCreatePlist();

		// get information to obtain memory dataspace.
		int rank = filespace->getSimpleExtentNdims();
		filespace->getSimpleExtentDims(dimsr);
		H5::DataSpace* memspace = new H5::DataSpace(rank, dimsr, NULL);

		length = dimsr[0];

		// load labels and channels
		if (whichCh == -1) {
			label = new int[length];
			dataset->read(label, H5::PredType::NATIVE_INT, *memspace, *filespace);
		} else {
			ch[whichCh] = new float[length * dimsr[1]];
			dataset->read(ch[whichCh], H5::PredType::NATIVE_FLOAT, *memspace,
				*filespace);
		}

		prop.close();
		delete filespace;
		delete memspace;

		return dataset;
	} catch (H5::FileIException error) {
		error.printError();
		return NULL;
	} catch (H5::DataSetIException error) {
		error.printError();
		return NULL;
	} catch (H5::DataSpaceIException error) {
		error.printError();
		return NULL;
	}
}


================================================
FILE: image/h5Data.hpp
================================================
#ifndef H5DATA_H
#define H5DATA_H

#include <iostream>
#include <string>

// hdf5 includes
#include "H5Cpp.h"

// h5 tags
namespace {
	const H5std_string datasetNameLa = "label";
	const H5std_string datasetNameCh = "ch";

	const hsize_t dimsLa[2] = {1, 1};
	const hsize_t dimsCh[2] = {1, 32 * 32};

	const int backgroundClass = 0;
}

#endif

// data class for 5f file
class h5Data {
private:
	std::string fileName;
	int numChannel;
	int length;

	// store dataset
	H5::H5File *dataFile = NULL;
	H5::DataSet *datasetLa = NULL;
	H5::DataSet *datasetCh[8];

	// store label and channels data
	int *label;
	float *ch[8];

public:
	h5Data(std::string f, int n);
	~h5Data();

	H5::DataSet* load(H5std_string datasetName, int whichCh);

	inline float* getCh(int whichCh) { return ch[whichCh]; }
	inline int* getLa() { return label; }
	inline int getLength() { return length; }
};


================================================
FILE: image/main.cpp
================================================
#include <vector>
#include <string>

// boost options
#include <boost/program_options.hpp>
namespace po = boost::program_options;

#include "seq.hpp"

int main(int argc, char *argv[]) {
	try {
		// generate argument parser
		po::options_description desc("Image: allowed options");
		desc.add_options()
			("help", "Print help messages")
			("source", po::value<std::string>(), "h5 source file")
			("target", po::value<std::string>(), "Flow target directory")
			("channel", po::value<int>(), "Channel number")
			("origin", po::value<int>(), "Origin size")
			("out", po::value<int>(), "Output size")
			("label", po::value<bool>(), "Print label in output file name")
			("bound", po::value<double>(), "Bound for normalization");

		po::variables_map vm;
		po::store(po::parse_command_line(argc, argv, desc), vm);
		po::notify(vm);

		std::string source, target;
		int numChannel = 1, origin = 32, out = 32;
		bool label = false;
		double bound = 15.0;

		// assign argument to local variables
		if (vm.count("help")) {
			std::cout << "Image" << std::endl << desc << std::endl;
			return 0;
		} else {
			if (vm.count("source")) source = vm["source"].as<std::string>();
			if (vm.count("target")) target = vm["target"].as<std::string>();
			if (vm.count("channel")) numChannel = vm["channel"].as<int>();
			if (vm.count("origin")) origin = vm["origin"].as<int>();
			if (vm.count("out")) out = vm["out"].as<int>();
			if (vm.count("label")) label = vm["label"].as<bool>();
			if (vm.count("bound")) bound = vm["bound"].as<double>();

			// generate flow for different channels
			Seq seq = Seq(source, target, numChannel, label);
			for (int i = 0; i < numChannel; i++) {
				seq.flow(i, origin, origin, out, out, bound);
			}
		}
	} catch (std::exception& e) {
			std::cerr << "Error: " << e.what() << std::endl;
			return 1;
	} catch (...) {
			std::cerr << "Exception of unknown type" << std::endl;
			return 1;
	}

	return 0;
}


================================================
FILE: image/seq.cpp
================================================
#include "seq.hpp"

Seq::Seq(std::string s, std::string t, int n, bool l) :
		source(s), target(t), numChannel(n), printLabel(l) {
	// read h5 file and get the data
	h5file = new h5Data(source, numChannel);
	label = h5file->getLa();
	for (int i = 0; i < numChannel; i++) {
		ch[i] = h5file->getCh(i);
	}
	length = h5file->getLength();
}

Seq::~Seq() {
	// delete files with pointer at the end, release channel data in the end
	delete h5file;
}

// convert float (0-1) value images to int (0-255) images with bound limit
void Seq::float2Image(const cv::Mat &floatMat, cv::Mat &imageMat,
		double l, double u) {
	for (int i = 0; i < floatMat.rows; i++) {
		for (int j = 0; j < floatMat.cols; j++) {
			float x = floatMat.at<float>(i, j);
			imageMat.at<uchar>(i, j) = cast(x, l, u);
		}
	}
}

// load array data image with offset to int images with resize
cv::Mat Seq::loadOrigin(float *image, int originHeight, int originWidth,
		int outHeight, int outWidth) {
	// convert array data from float to int image
	cv::Mat originFloat = cv::Mat(originHeight, originWidth, CV_32F, image);
	cv::Mat originInt = cv::Mat(cv::Size(originWidth, originHeight), CV_8UC1);
	float2Image(originFloat, originInt, 0.0, 1.0);

	// resize image if necessary
	cv::Mat resize = cv::Mat(cv::Size(outWidth, outHeight), CV_8UC1);
	if (originHeight == outHeight && originWidth == outWidth) {
		originInt.copyTo(resize);
	} else {
		cv::resize(originInt, resize, cv::Size(outWidth, outHeight),
			0, 0, cv::INTER_LINEAR);
	}

	return resize;
}

// generate flow and original image
void Seq::flow(int whichCh, int originHeight, int originWidth,
		int outHeight, int outWidth, double bound) {
	// following frames (start with one to support original flow computation)
	for (int i = 1; i < length; i++) {
		// load next image
		std::string imageName;
		if (printLabel) {
			imageName = target + "/ch" + std::to_string(whichCh) + "_" +
				std::to_string(i - 1) + "_" + std::to_string(label[i - 1]) + "_";
		} else {
			imageName = target + "/ch" + std::to_string(whichCh) + "_" +
				std::to_string(i - 1) + "_";
		}

		// write the original image and save to file
		cv::Mat image = loadOrigin(ch[whichCh] + i * originHeight * originWidth,
			originHeight, originWidth, outHeight, outWidth); // resize image for out
		cv::imwrite(imageName + "image.jpg", image);
	}
}


================================================
FILE: image/seq.hpp
================================================
#ifndef SEQ_H
#define SEQ_H

#include <iostream>
#include <string>

// opencv
#include <opencv2/opencv.hpp>

#include "h5Data.hpp"

#endif

class Seq {
private:
	std::string source, target;
	int numChannel;
	int length;
	bool printLabel;

	// use file pointer to avoid call destructor of h5Data too soon
	h5Data *h5file = NULL;

	// not in charge of manage memory
	int *label;
	float *ch[8];

	void resize();
	// void flow();
public:
	Seq(std::string s, std::string t, int n, bool l);
	~Seq();

	inline float cast(float v, float l, float u) {
		if (v > u) return 255;
		else if (v < l) return 0;
		else return cvRound(255 * (v - l) / (u - l));
	}

	void float2Image(const cv::Mat &floatMat, cv::Mat &imageMat,
		double lBound, double uBound);
	cv::Mat loadOrigin(float *image, int originHeight, int originWidth,
		int outHeight, int outWidth);

	void flow(int whichCh, int originHeight, int originWidth,
		int outHeight, int outWidth, double bound);
};


================================================
FILE: net/data.lua
================================================
require 'class'
require 'paths'
require 'mattorch'
local json = require 'lunajson'

require 'net.util'

local Data = torch.class('Data')

function Data:__init(fileDir, listFile, meanFile, listType,
    inputSize, inputCh, labelSize, dataSize, dataCh, ch, useCuda)
  self.seqList = self:getSeq(fileDir, listFile, listType)
  self.mean = self:loadMean(meanFile)
  self.inputSize = inputSize
  self.inputCh = inputCh
  self.labelSize = labelSize
  self.dataSize = dataSize
  self.dataCh = dataCh
  self.ch = ch
  self.useCuda = useCuda
end

function Data:getSeq(fileDir, listFile, listType)
  -- read file into json
  local f = io.open(listFile, 'rb')
  local jsonString = f:read('*all')
  local jsonObject = json.decode(jsonString)
  f:close()
  -- arrange it according to different label
  local seqList = {}
  for i in pairs(jsonObject[listType]) do
    seqList[#seqList + 1] = paths.concat(fileDir, jsonObject[listType][i])
  end
  return seqList
end

-- random shuffle a list
function Data:shuffleList(list)
  local n, random = #list, math.random
  for i = 1, n do
    local j, k = random(n), random(n)
    list[j], list[k] = list[k], list[j]
  end
  return list
end

-- load mat mean file
function Data:loadMean(meanFile)
  return mattorch.load(meanFile)
end

function Data:initBatch(batchSize, seqLength)
  -- initialize inputs and targets, zero mask will ignore null target
  local inputs, targets = {}, {}
  for l = 1, seqLength do
    -- init with 4d data (cnn and uni)
    inputs[l] = torch.Tensor(batchSize,
      self.inputCh, self.inputSize, self.inputSize):zero()
    targets[l] = torch.Tensor(batchSize):fill(self.labelSize)
  end
  return inputs, targets
end


================================================
FILE: net/imageseq.lua
================================================
require 'class'
require 'paths'
require 'image'
local json = require 'lunajson'

require 'net.seq'
require 'net.util'

local ImageSeq = torch.class('ImageSeq', 'Seq')

function ImageSeq:__init(seqName, mean, dataSize, dataCh, ch, useCuda)
  Seq.__init(self, seqName, mean, dataSize, dataCh, ch, useCuda)
  self.info = self:getSeqInfo()
  -- load frame depends on image type, lib image support only float tensor
  -- reset tensor type back to float tensor when using cuda
  torch.setdefaulttensortype('torch.FloatTensor')
  self.imageFrame = self:loadFrame()
  -- set tensor type back to cuda tensor when using cuda
  if self.useCuda then
    torch.setdefaulttensortype('torch.CudaTensor')
  end
end

-- read label file
function ImageSeq:getSeqInfo()
  -- read file into json
  local f = io.open(paths.concat(self.name, 'label.json'), 'rb')
  local jsonString = f:read('*all')
  local jsonObject = json.decode(jsonString)
  f:close()
  -- convert into num based index, 0-based to 1-based
  -- label is also 1-based: 0-4 => 1-5
  local seqInfo = {}
  for i in pairs(jsonObject) do
    seqInfo[i + 1] = jsonObject[i] + 1
  end
  return seqInfo
end

function ImageSeq:loadFrame()
  local frame = {}
  local m = self.mean[string.format('ch%d_image', self.ch)]
    :permute(3, 2, 1) -- the matlab order is reversed
  -- access each frame, 1-based to 0-based image name
  for f = 1, #self.info do
    local name = string.format('ch%d_%d_image.jpg', self.ch, f - 1)
    -- for image tensor, first map from 0-1 to 0-255,
    -- then minus mean (also in 0-255)
    frame[#frame + 1] = (image.load(paths.concat(self.name, name), 1) * 256) - m
  end
  return frame
end

function ImageSeq:getImage(f)
  -- appending later frames
  local mat = self.imageFrame[f]
  for i = 1, self.dataCh - 1 do
    mat = torch.cat(mat, self.imageFrame[math.min(#self.info, f + i)], 1)
  end
  return mat
end

function ImageSeq:getFrame(f)
  -- load data depends on image or flow
  local matrix = self:getImage(f)
  return matrix
end


================================================
FILE: net/main.lua
================================================
require 'net.util'

-- parse cmd parameters
local cmd = torch.CmdLine()

cmd:option('--file', '../collect_image/origin_trans', 'Image dir')
cmd:option('--list', 'tmp/file.json', 'List file json')
cmd:option('--load', '', 'Load model file')

cmd:option('--inputsize', 32, 'Input size')
cmd:option('--inputch', 4, 'Input channel')
cmd:option('--label', 3, 'Label size')

cmd:option('--datasize', 32, 'Data size')
cmd:option('--datach', 4, 'Data channel (number of stack)')

cmd:option('--batch', 4, 'Batch size')
cmd:option('--maxseq', 40, 'Sequence length')

cmd:option('--cuda', false, 'Use CUDA')
cmd:option('--cudnn', false, 'Use CUDNN')
local opt = cmd:parse(arg or {})

--- other parameters
local meanFile = 'tmp/mean_32.mat'
local ch = 1

-- default tensor
if opt.cuda then
  require 'cutorch'
  require 'cunn'
  torch.setdefaulttensortype('torch.CudaTensor')
else
  torch.setdefaulttensortype('torch.FloatTensor')
end

-- train net, switch train type
local trainObject
require 'net.rnntrain'
trainObject = RnnTrain(opt.file, opt.list, meanFile,
  opt.inputsize, opt.inputch, opt.label,
  opt.datasize, opt.datach, ch,
  opt.load, opt.cuda, opt.cudnn)

trainObject:train(opt.batch, opt.maxseq)


================================================
FILE: net/net.lua
================================================
require 'class'

local Net = torch.class('Net')

function Net:__init(useCuda, useCudnn)
  -- cuda support
  self.useCuda = useCuda
  self.useCudnn = useCudnn
  if self.useCuda then
    require 'cutorch'
    require 'cunn'
    if self.useCudnn then
      require 'cudnn'
    end
  end
end

-- load already exist net
function Net:loadNet(loadFile)
  print('[net] loading model ' .. loadFile)
  net = torch.load(loadFile)
  print(net)
  return net
end

-- cuda and cudnn support
function Net:cudaNet(net)
  if self.useCuda then
    print('[net] with cuda')
    net = net:cuda()
    if self.useCudnn then
      print('[net] with cudnn')
      cudnn.convert(net, cudnn)
    end
  else
    print('[net] without cuda')
  end
  print(net)
  return net
end


================================================
FILE: net/rnndata.lua
================================================
require 'class'

require 'net.data'
require 'net.imageseq'

local RnnData = torch.class('RnnData', 'Data')

function RnnData:__init(fileDir, listFile, meanFile, listType,
    inputSize, inputCh, labelSize, dataSize, dataCh, ch, useCuda)
  Data.__init(self, fileDir, listFile, meanFile, listType,
    inputSize, inputCh, labelSize, dataSize, dataCh, ch, useCuda)
end

function RnnData:frameSeq(seq, maxSeq)
  -- load label for each frame, label is 1-based, f is frame id
  -- offset introduce zero paddings at the beginning of the sequence
  local frames = {}
  if #seq.info < maxSeq then
    local offset = maxSeq - #seq.info
    for f = 1, #seq.info do
      frames[offset + f] = f
    end
  else
    for f = 1, maxSeq do
      frames[f] = math.floor(f * (#seq.info / maxSeq))
    end
  end
  return frames
end

-- load batch to tensor
function RnnData:loadBatch(batchNum, batchSize, maxSeq)
  local inputs, targets = self:initBatch(batchSize, maxSeq)
  -- load data into inputs and targets
  local batchStart = (batchNum - 1) * batchSize
  for b = 1, batchSize do
    -- get seq info
    local seq
    seq = ImageSeq(self.seqList[batchStart + b], self.mean,
      self.dataSize, self.dataCh, self.ch, self.useCuda)
    -- generate frame sequence
    local frames = self:frameSeq(seq, maxSeq)
    for i, v in pairs(frames) do
      inputs[i][b] = seq:getFrame(v)
      targets[i][b] = seq.info[v]
    end
  end

  return inputs, targets
end


================================================
FILE: net/rnntrain.lua
================================================
require 'class'

require 'net.train'
require 'net.util'

local RnnTrain = torch.class('RnnTrain', 'Train')

function RnnTrain:__init(fileDir, listFile, meanFile,
    inputSize, inputCh, labelSize, dataSize, dataCh, ch,
    loadFile, useCuda, useCudnn)
  -- call base train constructor
  Train.__init(self, labelSize, useCuda, useCudnn)
  -- load data
  require 'net.rnndata'
  self.evalData = RnnData(fileDir, listFile, meanFile, 'eval',
    inputSize, inputCh, labelSize, dataSize, dataCh, ch, useCuda)
  print(string.format('[eval] data with %d seq', #self.evalData.seqList))
  -- build net
  local netObject
  require 'net.uninet'
  netObject = UniNet(useCuda, useCudnn)
  self.net = netObject:loadNet(loadFile)
end

function RnnTrain:batchEval(b, inputs)
  -- evaluate forward as whole sequence
  self.net:evaluate()
  local outputs = self.net:forward(self:convertCuda(inputs))
  self.net:forget()
  return outputs
end


================================================
FILE: net/seq.lua
================================================
require 'class'
require 'paths'
require 'image'
local json = require 'lunajson'

require 'net.util'

local Seq = torch.class('Seq')

function Seq:__init(seqName, mean, dataSize, dataCh, ch, useCuda)
  -- load info
  self.name = seqName
  self.mean = mean
  self.dataSize = dataSize
  self.dataCh = dataCh
  self.ch = ch
  self.useCuda = useCuda
end


================================================
FILE: net/stat.lua
================================================
require 'class'
local json = require 'lunajson'

local Stat = torch.class('Stat')

function Stat:__init(labelSize, batchSize, maxSeq)
  self.labelSize = labelSize
  self.batchSize = batchSize
  self.maxSeq = maxSeq
  -- init confusion matrix (correct label (size) * predict label (size))
  self.confus = self:initMat(labelSize, labelSize)
end

function Stat:initMat(height, width)
  local mat = {}
  for h = 1, height do
    mat[h] = {}
    for w = 1, width do
      mat[h][w] = 0
    end
  end
  return mat
end

function Stat:updateStat(outputs, targets)
  -- calculate one sequence by one sequence
  for o = 1, #outputs do
    local _, indices = torch.max(outputs[o], 2)
    for b = 1, self.batchSize do
      local right, pred = targets[o][b], indices[b][1]
      -- update confusion matrix
      self.confus[right][pred] = self.confus[right][pred] + 1
    end
  end
end

function Stat:sum(list, length)
  local result = 0
  -- ignore null label in sum
  for i = 1, length do
    result = result + list[i]
  end
  return result
end

-- calculate per label stat
function Stat:calPerLabel()
  local right, count = {}, {}
  -- iterate for all label
  for l = 1, self.labelSize do
    right[l] = self.confus[l][l]
    count[l] = self:sum(self.confus[l], #self.confus[l])
  end
  return right, count
end

-- print matrix
function Stat:printMat(str, right, count, matRight, matCount)
  for l = 1, #right do
    if count[l] > 0 then
      local accLine = right[l] * 100 / count[l]
      io.write(string.format('    %s %02d: %02.0f [', str, l, accLine))
      for ll = 1, #matRight[1] do
        -- print each element depends on the type and value
        if type(matCount[l]) == 'number' and matCount[l] > 0 then
          local acc = matRight[l][ll] * 100 / matCount[l]
          io.write(string.format(' %02.0f ', acc))
        else
          io.write(' -- ')
        end
      end
      io.write(']\n')
    else
      io.write(string.format('    %s %02d: --\n', str, l))
    end
  end
end

function Stat:print(type)
  local right, count = self:calPerLabel()
  -- calculate sum of label
  local accEpoch = self:sum(right, #right - 1) / self:sum(count, #count - 1)
  io.write(string.format('[%s] accuracy %f\n', type, accEpoch))
  -- print per label and confusion matrix
  self:printMat('label', right, count, self.confus, count)
  -- return accuracy for lr update
  return accEpoch
end


================================================
FILE: net/train.lua
================================================
require 'class'

local Train = torch.class('Train')

function Train:__init(labelSize, useCuda, useCudnn)
  -- require and cuda support
  require 'net.stat'
  self.useCuda = useCuda
  self.useCudnn = useCudnn
  if useCuda then
    require 'cutorch'
    require 'cunn'
  end
  -- other parameters
  self.labelSize = labelSize
end

-- one step of evaluate to all data
function Train:epochEval(batchSize, maxSeq)
  local stat = Stat(self.labelSize, batchSize, maxSeq)
  for b = 1, math.floor(#self.evalData.seqList / batchSize) do
    -- load data, start evaluating, depends on net type
    local inputs, targets = self.evalData:loadBatch(b, batchSize, maxSeq)
    local outputs = self:batchEval(b, inputs)
    -- update confusion matrix
    stat:updateStat(outputs, targets)
  end
  return stat:print('eval')
end

-- convert to cuda
function Train:convertCuda(list)
  if self.cuda then
    -- convert table one by one to cuda
    for l = 1, #list do
      list[l] = list[l]:cuda()
    end
    return list
  else
    return list
  end
end

function Train:train(batchSize, maxSeq)
  -- evaluate by epoch
  self:epochEval(batchSize, maxSeq)
  print('Finished')
end


================================================
FILE: net/uninet.lua
================================================
require 'class'
require 'rnn'

require 'net.net'

local UniNet = torch.class('UniNet', 'Net')

function UniNet:__init(useCuda, useCudnn)
  Net.__init(self, useCuda, useCudnn)
end


================================================
FILE: net/util.lua
================================================
-- helper function to check if string start with some sub-string
function string:startsWith(start)
  return string.sub(self, 1, string.len(start)) == start
end

function string:split(sep)
  local sep, fields = sep or ":", {}
  local pattern = string.format("([^%s]+)", sep)
  self:gsub(pattern, function(c) fields[#fields+1] = c end)
  return fields
end


================================================
FILE: pre/image.py
================================================
import cv2
import numpy as np

class image:
    def __init__(self, file_name):
        self.file_name = file_name

    # load image and resize image with size
    def load(self, height, width):
        frame = cv2.imread(self.file_name, cv2.IMREAD_GRAYSCALE)
        if height and width:
            frame = cv2.resize(frame, (width, height))
        frame = np.expand_dims(frame, axis=0).astype('float32')
        return frame


================================================
FILE: pre/main.py
================================================
import argparse

from seq import Seq

# call different method depends on op
def main(op, file_dir, target_dir, num_channel=1, redo=False,
        origin_size=32, out_size=32):
    s = Seq()

    if op == 'image':
        s.generate_image(file_dir, target_dir, num_channel,
            origin_size, out_size, redo)
    elif op == 'clean':
        s.clean_image(file_dir)
    elif op == 'mean':
        s.generate_mean(file_dir, target_dir, num_channel, out_size)
    else:
        raise NameError('The operation type {} is undefined'.format(op))

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Generate image")
    parser.add_argument('--op', type=str)
    parser.add_argument('--file', type=str)
    parser.add_argument('--target', type=str)
    parser.add_argument('--channel', type=int)
    parser.add_argument('--redo',  action='store_true')
    parser.add_argument('--originsize', type=int)
    parser.add_argument('--outsize', type=int)

    args = parser.parse_args()

    main(args.op, args.file, args.target, args.channel, args.redo,
        args.originsize, args.outsize)


================================================
FILE: pre/op.py
================================================
import os, subprocess
import numpy as np
import h5py
import json

from image import image

class Operation:
    def __init__(self):
        pass

    # generate image for each sequence
    def generate_image(self, source, target, num_channel, origin_size=32,
            out_size=32, bound=15.0, bin_image='image/bin/image'):
        if not os.path.exists(target): os.makedirs(target)
        # run flow command
        p = subprocess.Popen([bin_image,
            '--source', source, '--target', target,
            '--channel', str(num_channel),
            '--origin', str(origin_size),
            '--out', str(out_size),
            '--bound', str(bound)])
        p.wait()
        # generate label file
        self._generate_label(source, target)

    # load labels
    def _generate_label(self, source, target, label_file='label.json'):
        with h5py.File(source, 'r') as hf:
            labels = [int(l) for l in hf['label'][()]]
        # save labels as a dict in json file
        labels_dict = {i: l
            for i, l in zip(range(len(labels) - 1), labels[1:])}
        with open(os.path.join(target, label_file), 'w') as jf:
            json.dump(labels_dict, jf)

    # delete generated image files
    def clean_image(self, file_dir, label_file='label.json'):
        print 'Cleaning', file_dir
        # delete images in dir
        image_files = [os.path.join(file_dir, f)
            for f in os.listdir(file_dir)
            if os.path.isfile(os.path.join(file_dir, f)) and 'jpg' in f]
        for image_file in image_files:
            try:
                os.remove(image_file)
            except Exception, e:
                print e
        # delete label file
        try:
            os.remove(os.path.join(file_dir, label_file))
        except Exception, e:
            print e
        # delete dir
        try:
            os.rmdir(file_dir)
        except Exception, e:
            print e

    # get frame count
    def _get_frame_num(self, source, label_file='label.json'):
        with open(os.path.join(source, label_file), 'r') as jf:
            labels_dict = json.load(jf)
        return len(labels_dict)

    # setup mean counter and accumulator at the beginning
    def setup_mean(self, num_channel, out_size):
        self.image_sums = {}
        self.count = 0.0
        for c in range(num_channel):
            self.image_sums['ch%i_image' % (c,)] = \
                np.zeros((1, out_size, out_size), dtype='float32')

    # accumulate mean for each sequence
    def accum_mean(self, source, num_channel, out_size):
        print 'Loading mean', source
        frame_num = self._get_frame_num(source)
        self.count += frame_num
        for c in range(num_channel):
            for i in range(frame_num):
                image_name = os.path.join(source,
                    'ch%i_%i_image.jpg' % (c, i))
                self.image_sums['ch%i_image' % (c,)] += \
                    image(image_name).load(out_size, out_size)

    # save accumulated mean to file
    def save_mean(self, mean_file, num_channel):
        # store file as hdf5
        if mean_file.endswith('h5'):
            print 'Save as hdf5'
            with h5py.File(mean_file, 'w') as f:
                for c in range(num_channel):
                    f.create_dataset('ch%i_image' % (c,),
                        data=self.image_sums['ch%i_image' % (c,)]
                            / self.count)
        # store file as matlab data
        elif mean_file.endswith('mat'):
            import scipy.io as sio
            print 'Save as mat'
            data = {}
            for c in range(num_channel):
                data['ch%i_image' % (c,)] = \
                    self.image_sums['ch%i_image' % (c,)] / self.count
            sio.savemat(mean_file, data)


================================================
FILE: pre/seq.py
================================================
import os, re
import numpy as np
import h5py

from op import Operation

class Seq:
    def __init__(self):
        pass

    # get files in label directories
    def get_h5(self, file_dir):
        return [os.path.join(file_dir, f)
            for f in os.listdir(file_dir)
            if os.path.isfile(os.path.join(file_dir, f)) and 'h5' in f]

    # get list of target image dir names
    def get_dir(self, file_dir):
        return [os.path.join(file_dir, d)
            for d in os.listdir(file_dir)
            if os.path.isdir(os.path.join(file_dir, d))]

    # generate image
    def generate_image(self, file_dir, target_dir, num_channel,
            origin_size, out_size, redo):
        source_files = self.get_h5(file_dir)
        o = Operation()
        for s in source_files:
            t = os.path.join(target_dir, re.sub('.h5', '',
                os.path.basename(s)))
            if redo or not os.path.exists(t):
                print 'Generating', s
                o.generate_image(s, t, num_channel, origin_size, out_size)

    # clean image
    def clean_image(self, file_dir):
        file_dirs = self.get_dir(file_dir)
        o = Operation()
        for d in file_dirs:
            o.clean_image(d)

    # generate mean
    def generate_mean(self, file_dir, target, num_channel, out_size):
        o = Operation()
        o.setup_mean(num_channel, out_size)
        source_dir = self.get_dir(file_dir)
        for s in source_dir:
            o.accum_mean(s, num_channel, out_size)
        o.save_mean(target, num_channel)