Repository: simonwsw/deep-soli
Branch: master
Commit: e748041d59bb
Files: 26
Total size: 61.3 KB
Directory structure:
gitextract_2w_qnpkh/
├── .gitignore
├── LICENSE
├── README.md
├── cite.bib
├── config/
│ └── file_half.json
├── image/
│ ├── Makefile
│ ├── h5Data.cpp
│ ├── h5Data.hpp
│ ├── main.cpp
│ ├── seq.cpp
│ └── seq.hpp
├── net/
│ ├── data.lua
│ ├── imageseq.lua
│ ├── main.lua
│ ├── net.lua
│ ├── rnndata.lua
│ ├── rnntrain.lua
│ ├── seq.lua
│ ├── stat.lua
│ ├── train.lua
│ ├── uninet.lua
│ └── util.lua
└── pre/
├── image.py
├── main.py
├── op.py
└── seq.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
# Mac
.DS_Store
# Matlab
*.m~
*.asv
# Vim
*~
# Python binary
*.pyc
# Temp dir
tmp
# Build and CMake
build
# openFrameworks
bin
obj
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2016 Saiwen Wang
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: README.md
================================================
# deep-soli
Gesture Recognition Using Neural Networks with Google's Project Soli Sensor
## Update
Dataset and trained model are now available.
## Introduction
This is the open source evaluation code base of our paper:
**Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition
in the Radio-Frequency Spectrum**
Saiwen Wang, Jie Song, Jamie Lien, Poupyrev Ivan, Otmar Hilliges
(link to the [paper](http://bit.ly/2ftSRcn))
Please cite the paper if you find the code useful. Thank you.
([BibTex](https://github.com/simonwsw/deep-soli/blob/master/cite.bib))
This project uses Google's [Project Soli](atap.google.com/soli) sensor.
> Soli is a new sensing technology that uses miniature radar to
> detect touchless gesture interactions.
>
> 
>
> Soli sensor technology works by emitting electromagnetic waves in a
> broad beam. Objects within the beam scatter this energy, reflecting
> some portion back towards the radar antenna. Properties of the
> reflected signal, such as energy, time delay, and frequency shift
> capture rich information about the object’s characteristics and
> dynamics, including size, shape, orientation, material, distance,
> and velocity.
Our paper uses a light-weight end-to-end trained Convolutional Neural Networks
and Recurrent Neural Networks architecture, recognizes 11 in-air gestures
with 87% per-frame accuracy, and can perform realtime predictions at 140Hz
on commodity hardware. (link to the [paper video](http://bit.ly/2fDd9iJ))
## Pre-request
- Python 2: HDF5, OpenCV 2 interfaces for python.
- C++: HDF5, OpenCV 2, Boost
- Lua JIT and Torch 7.
- Torch 7 packages: `class`, GPU support `cunn` and `cutorch`, Matlab
support `mattorch`, JSON support `lunajson`, Torch image library `image`
- Please note that `mattorch` is an outdated packages which is no
longer maintained.
## Quick start
- Make image binary
```
cd image
mkdir bin
make
```
- Preprocessing (HDF5 to images):
```
python pre/main.py --op image --file [dataset folder]
--target [target image folder] --channel 4 --originsize 32 --outsize 32
```
- Preprocessing (generate mean file):
```
python pre/main.py --op mean --file [image folder]
--target [mean file name] --channel 4 --outsize 32
```
- Load model and evaluate:
```
th net/main.lua --file [image folder] --list [train/test sequence split file]
--load [model file] --inputsize 32 --inputch 4 --label 13 --datasize 32
--datach 4 --batch 16 --maxseq 40 --cuda --cudnn
```
## Dataset
- Download [dataset](https://polybox.ethz.ch/index.php/s/wG93iTUdvRU8EaT)
(please let me know when the link doesn't work).
- Train/test split file (in JSON format) we used is stored in the repo
`config/file_half.json`.
- The dataset contains multiple preprocessed Range-Doppler Image sequences.
Each sequence is saved as a single HDF5 format data file. File names are
defined as `[gesture ID]_[session ID]_[instance ID].h5`. Range-Doppler Image
data of a specific channel can be accessed by dataset name `ch[channel ID]`.
Label can be accessed by dataset name `label`. Range-Doppler Image
data array has shape of `[number of frame] * 1024` (can be reshape back to 2D Range-Doppler Image to `32 * 32`)
- Simple Python code to access the data:
```python
# Demo code to extract data in python
import h5py
use_channel = 0
with h5py.File(file_name, 'r') as f:
# Data and label are numpy arrays
data = f['ch{}'.format(use_channel)][()]
label = f['label'][()]
```
- Dataset session arrangement for evaluation.
- 11 (gestures) * 25 (instances) * 10 (users) for cross user evaluation:
session 2 (25), 3 (25), 5 (25), 6 (25), 8 (25), 9 (25), 10 (25), 11 (25),
12 (25), 13 (25).
- 11 (gestures) * (50 (instances) * 4 (sessions) +
25 (instances) * 2 (sessions)) for single user
cross session evaluation: session 0 (50), 1 (50), 4 (50), 7 (50),
13 (25), 14 (25).
- Please refer to the paper for the gesture collecting
campaign details.
- The gestures are listed in the table below. Each column represents
one gesture and we snapshot three important steps for each gestures.
The gesture label is indicated by the number in the circle above. Please
notice that the gesture label order is different than the paper, as
we regroup gestures in the paper. Sequences with gesture ID 11 are
background signals with no presence of hand.
## Pre-trained model
- Download [model](https://polybox.ethz.ch/index.php/s/0SEdZqkn433dbEh)
- Trained proposed model, please refer to the paper for model detail.
- Simple Lua (Torch 7) code to load the model:
```
loadFile = 'uni_image_np_50.t7'
net = torch.load(loadFile)
print(net)
```
- The model uses layers support `cudnn`.
## Comments on the code base
This is a simplified version of the original code base we used for all the
experiments in the paper. The complex Torch based class hierarchies in
the `net` reflects varies model architectures we tried during the
experiments. For simplicity, we only make the evaluation part public.
The model detail can be found both in the paper and the model file.
## License
This project is licensed under the terms of the MIT license.
================================================
FILE: cite.bib
================================================
@inproceedings{wang2016interacting,
title={Interacting with soli: Exploring fine-grained dynamic gesture recognition in the radio-frequency spectrum},
author={Wang, Saiwen and Song, Jie and Lien, Jaime and Poupyrev, Ivan and Hilliges, Otmar},
booktitle={Proceedings of the 29th Annual Symposium on User Interface Software and Technology},
pages={851--860},
year={2016},
organization={ACM}
}
================================================
FILE: config/file_half.json
================================================
{"train": ["0_8_4", "0_3_1", "0_2_0", "0_13_5", "0_2_19", "0_3_2", "0_12_1", "0_9_7", "0_3_22", "0_2_2", "0_2_3", "0_5_8", "0_8_21", "0_9_4", "0_2_17", "0_13_13", "0_2_22", "0_9_15", "0_9_17", "0_11_14", "0_10_14", "0_2_16", "0_6_7", "0_2_10", "0_12_19", "0_8_24", "0_8_3", "0_11_4", "0_2_11", "0_3_3", "0_12_14", "0_5_7", "0_11_3", "0_10_1", "0_9_14", "0_11_23", "0_12_13", "0_8_19", "0_8_16", "0_10_16", "0_9_2", "0_8_13", "0_3_9", "0_9_12", "0_13_12", "0_5_15", "0_6_15", "0_6_0", "0_5_0", "0_10_7", "0_9_3", "0_3_11", "0_3_6", "0_13_18", "0_6_19", "0_9_5", "0_10_11", "0_12_7", "0_12_16", "0_11_16", "0_8_14", "0_11_20", "0_10_0", "0_10_13", "0_6_17", "0_6_11", "0_11_17", "0_10_2", "0_11_2", "0_12_18", "0_12_4", "0_11_9", "0_2_14", "0_6_6", "0_5_9", "0_10_8", "0_8_23", "0_11_0", "0_3_20", "0_11_7", "0_11_13", "0_2_7", "0_11_5", "0_8_12", "0_10_21", "0_2_9", "0_12_5", "0_3_23", "0_3_19", "0_8_7", "0_10_10", "0_5_2", "0_13_22", "0_8_5", "0_11_22", "0_11_24", "0_9_24", "0_3_12", "0_2_6", "0_8_20", "0_5_18", "0_5_3", "0_3_8", "0_9_9", "0_11_19", "0_13_23", "0_6_5", "0_11_18", "0_3_13", "0_3_16", "0_9_1", "0_3_4", "0_5_6", "0_6_22", "0_13_20", "0_9_10", "0_3_17", "0_6_10", "0_2_24", "0_10_3", "0_5_5", "0_6_18", "0_6_4", "0_2_1", "0_3_14", "0_10_5", "1_8_3", "1_2_24", "1_2_16", "1_9_12", "1_6_14", "1_13_24", "1_3_6", "1_3_1", "1_9_7", "1_3_24", "1_5_14", "1_9_6", "1_13_6", "1_13_3", "1_2_6", "1_6_10", "1_10_10", "1_2_12", "1_3_4", "1_5_12", "1_12_7", "1_12_24", "1_5_21", "1_12_12", "1_11_9", "1_11_10", "1_13_10", "1_11_19", "1_6_4", "1_5_1", "1_3_16", "1_5_6", "1_6_15", "1_5_18", "1_2_13", "1_9_21", "1_13_19", "1_3_0", "1_13_1", "1_10_12", "1_12_5", "1_9_13", "1_11_22", "1_3_10", "1_3_14", "1_8_11", "1_5_15", "1_6_2", "1_5_9", "1_9_24", "1_8_10", "1_5_10", "1_6_0", "1_10_8", "1_10_2", "1_8_19", "1_13_5", "1_12_11", "1_12_22", "1_9_17", "1_2_18", "1_11_4", "1_2_0", "1_5_22", "1_3_15", "1_10_20", "1_2_5", "1_2_8", "1_13_2", "1_10_5", "1_11_3", "1_10_24", "1_10_21", "1_6_11", "1_11_14", "1_9_1", "1_8_22", "1_13_21", "1_10_14", "1_9_19", "1_6_16", "1_2_10", "1_3_21", "1_8_14", "1_10_1", "1_12_14", "1_2_17", "1_5_19", "1_13_9", "1_8_12", "1_2_11", "1_13_11", "1_3_17", "1_11_17", "1_12_2", "1_12_15", "1_3_9", "1_10_6", "1_11_15", "1_2_23", "1_2_14", "1_8_16", "1_6_3", "1_3_5", "1_11_24", "1_13_14", "1_13_20", "1_3_2", "1_10_18", "1_9_16", "1_9_23", "1_8_2", "1_12_18", "1_3_8", "1_13_7", "1_2_4", "1_2_22", "1_5_8", "1_6_24", "1_2_9", "1_2_2", "1_11_12", "1_11_21", "1_11_8", "1_8_5", "1_11_18", "2_6_6", "2_9_20", "2_2_7", "2_3_16", "2_8_24", "2_8_9", "2_8_21", "2_11_17", "2_8_23", "2_9_8", "2_3_8", "2_2_11", "2_8_18", "2_13_11", "2_9_5", "2_11_14", "2_13_9", "2_12_14", "2_8_16", "2_10_21", "2_2_13", "2_13_18", "2_2_23", "2_12_15", "2_8_12", "2_11_0", "2_5_5", "2_5_19", "2_8_13", "2_5_12", "2_13_21", "2_3_13", "2_9_22", "2_2_9", "2_10_20", "2_6_2", "2_5_3", "2_10_17", "2_11_24", "2_10_12", "2_11_16", "2_11_5", "2_3_0", "2_12_19", "2_3_2", "2_10_0", "2_10_18", "2_13_2", "2_12_17", "2_12_11", "2_13_10", "2_5_17", "2_13_7", "2_11_8", "2_5_6", "2_13_20", "2_8_7", "2_6_7", "2_12_22", "2_6_16", "2_5_21", "2_3_5", "2_5_11", "2_13_17", "2_3_1", "2_6_12", "2_10_24", "2_3_3", "2_8_2", "2_10_23", "2_3_7", "2_5_1", "2_8_11", "2_5_15", "2_6_23", "2_2_5", "2_12_23", "2_5_23", "2_10_22", "2_2_3", "2_13_1", "2_9_7", "2_12_10", "2_12_9", "2_11_6", "2_2_18", "2_12_2", "2_10_10", "2_2_2", "2_11_2", "2_3_20", "2_13_6", "2_12_5", "2_11_18", "2_6_14", "2_5_20", "2_5_8", "2_8_3", "2_10_5", "2_8_4", "2_9_17", "2_2_14", "2_10_19", "2_2_22", "2_2_12", "2_11_21", "2_12_7", "2_6_5", "2_6_13", "2_12_24", "2_5_14", "2_6_8", "2_8_0", "2_8_1", "2_10_7", "2_12_3", "2_3_22", "2_9_24", "2_3_23", "2_12_20", "2_10_11", "2_3_19", "2_5_13", "2_11_9", "2_9_23", "2_2_20", "3_5_21", "3_8_14", "3_9_1", "3_10_21", "3_9_14", "3_12_1", "3_8_24", "3_3_4", "3_12_3", "3_9_24", "3_13_4", "3_2_2", "3_12_14", "3_8_4", "3_6_22", "3_10_16", "3_10_2", "3_5_4", "3_3_13", "3_2_21", "3_11_0", "3_6_16", "3_3_2", "3_8_12", "3_12_18", "3_5_2", "3_6_12", "3_11_13", "3_10_6", "3_6_8", "3_6_21", "3_10_19", "3_11_2", "3_3_5", "3_12_19", "3_13_18", "3_12_12", "3_13_22", "3_3_22", "3_10_8", "3_6_17", "3_11_10", "3_2_10", "3_5_14", "3_6_18", "3_2_9", "3_5_22", "3_9_3", "3_3_8", "3_12_20", "3_2_12", "3_10_1", "3_13_9", "3_13_12", "3_11_20", "3_12_10", "3_6_24", "3_13_2", "3_5_6", "3_12_21", "3_10_17", "3_10_18", "3_9_23", "3_3_12", "3_12_11", "3_8_17", "3_11_6", "3_8_8", "3_11_7", "3_3_19", "3_10_22", "3_10_14", "3_10_20", "3_13_15", "3_5_8", "3_3_10", "3_8_20", "3_11_9", "3_9_13", "3_6_11", "3_6_19", "3_11_18", "3_13_5", "3_8_9", "3_13_21", "3_9_21", "3_2_3", "3_9_7", "3_2_11", "3_3_20", "3_5_15", "3_12_7", "3_5_5", "3_3_17", "3_10_13", "3_11_19", "3_9_4", "3_3_14", "3_10_7", "3_9_11", "3_10_11", "3_10_10", "3_13_17", "3_5_19", "3_2_23", "3_6_5", "3_3_11", "3_13_14", "3_8_10", "3_5_11", "3_9_22", "3_13_13", "3_3_16", "3_11_5", "3_3_18", "3_6_23", "3_5_17", "3_13_1", "3_9_20", "3_5_24", "3_12_22", "3_6_6", "3_12_16", "3_12_15", "3_10_3", "3_5_16", "4_3_14", "4_3_7", "4_3_20", "4_3_5", "4_5_8", "4_2_15", "4_6_10", "4_13_9", "4_13_23", "4_10_16", "4_5_16", "4_10_3", "4_3_1", "4_10_24", "4_2_13", "4_5_11", "4_6_23", "4_9_1", "4_10_23", "4_6_20", "4_2_22", "4_11_18", "4_5_15", "4_9_22", "4_10_20", "4_11_7", "4_12_4", "4_3_0", "4_9_15", "4_9_17", "4_9_5", "4_9_21", "4_9_4", "4_13_24", "4_9_24", "4_10_11", "4_3_9", "4_6_2", "4_9_6", "4_3_3", "4_8_15", "4_3_18", "4_6_14", "4_13_11", "4_8_19", "4_5_4", "4_3_16", "4_6_6", "4_5_7", "4_8_14", "4_11_12", "4_12_14", "4_11_22", "4_11_17", "4_8_24", "4_6_1", "4_13_20", "4_12_21", "4_8_7", "4_3_22", "4_12_17", "4_10_9", "4_11_19", "4_3_17", "4_2_16", "4_10_19", "4_13_21", "4_9_20", "4_8_10", "4_5_2", "4_9_3", "4_9_16", "4_2_7", "4_13_3", "4_10_0", "4_8_22", "4_11_23", "4_5_0", "4_6_0", "4_11_6", "4_9_10", "4_2_6", "4_9_18", "4_8_23", "4_5_19", "4_2_14", "4_12_19", "4_11_4", "4_8_13", "4_8_2", "4_5_18", "4_13_15", "4_13_19", "4_8_5", "4_11_3", "4_3_19", "4_2_23", "4_2_8", "4_3_12", "4_6_18", "4_12_9", "4_2_4", "4_11_0", "4_12_5", "4_12_23", "4_6_15", "4_11_5", "4_3_13", "4_5_24", "4_12_18", "4_13_7", "4_5_12", "4_8_9", "4_6_8", "4_2_3", "4_10_17", "4_5_1", "4_6_24", "4_9_14", "4_3_8", "4_9_19", "4_10_18", "4_5_14", "4_10_1", "4_9_23", "4_12_13", "5_10_0", "5_3_24", "5_12_1", "5_12_4", "5_2_10", "5_8_22", "5_2_21", "5_9_7", "5_6_21", "5_12_14", "5_13_3", "5_12_5", "5_11_8", "5_10_11", "5_9_20", "5_11_1", "5_8_1", "5_12_9", "5_12_22", "5_5_17", "5_6_15", "5_9_23", "5_8_16", "5_6_19", "5_2_11", "5_13_4", "5_11_5", "5_3_20", "5_13_0", "5_8_13", "5_6_5", "5_2_19", "5_9_3", "5_5_10", "5_5_0", "5_9_16", "5_6_0", "5_2_15", "5_5_12", "5_2_4", "5_2_7", "5_3_2", "5_3_11", "5_11_11", "5_3_8", "5_11_13", "5_13_21", "5_5_15", "5_12_24", "5_3_7", "5_3_3", "5_5_21", "5_13_13", "5_10_9", "5_13_6", "5_9_17", "5_5_9", "5_10_20", "5_12_17", "5_11_18", "5_6_16", "5_9_11", "5_2_18", "5_2_9", "5_8_18", "5_13_18", "5_9_5", "5_3_18", "5_2_13", "5_12_3", "5_10_16", "5_3_13", "5_2_17", "5_10_8", "5_5_8", "5_13_2", "5_13_14", "5_13_5", "5_12_6", "5_6_1", "5_11_4", "5_3_5", "5_12_0", "5_6_2", "5_12_18", "5_6_13", "5_13_1", "5_9_13", "5_10_24", "5_3_6", "5_5_23", "5_12_15", "5_12_7", "5_13_9", "5_2_6", "5_10_14", "5_8_11", "5_10_15", "5_6_3", "5_9_0", "5_9_22", "5_8_10", "5_8_12", "5_8_19", "5_2_3", "5_2_2", "5_13_8", "5_10_2", "5_9_6", "5_8_4", "5_10_6", "5_6_14", "5_9_10", "5_3_21", "5_11_12", "5_8_5", "5_10_22", "5_13_12", "5_8_23", "5_5_4", "5_10_13", "5_5_19", "5_8_8", "5_6_17", "5_3_17", "5_5_11", "6_11_2", "6_11_21", "6_3_5", "6_6_19", "6_12_4", "6_3_24", "6_13_0", "6_6_16", "6_3_18", "6_3_13", "6_9_20", "6_9_24", "6_9_21", "6_2_12", "6_13_20", "6_6_22", "6_9_13", "6_8_3", "6_2_20", "6_2_4", "6_10_0", "6_9_6", "6_10_13", "6_3_11", "6_11_11", "6_8_13", "6_8_8", "6_9_9", "6_12_10", "6_10_7", "6_12_16", "6_3_14", "6_8_17", "6_6_11", "6_9_2", "6_8_0", "6_10_1", "6_5_5", "6_3_4", "6_2_16", "6_8_2", "6_6_2", "6_10_8", "6_9_3", "6_9_14", "6_10_12", "6_2_1", "6_9_0", "6_2_15", "6_9_15", "6_10_14", "6_5_13", "6_3_9", "6_12_24", "6_3_2", "6_6_10", "6_5_20", "6_9_22", "6_6_9", "6_9_1", "6_6_24", "6_13_7", "6_5_2", "6_10_20", "6_12_9", "6_10_3", "6_9_8", "6_2_8", "6_12_23", "6_10_5", "6_8_15", "6_13_12", "6_9_16", "6_10_10", "6_5_14", "6_13_1", "6_11_3", "6_13_13", "6_3_20", "6_9_11", "6_5_12", "6_12_11", "6_9_18", "6_6_21", "6_8_12", "6_5_10", "6_8_23", "6_3_6", "6_9_12", "6_8_9", "6_5_1", "6_6_20", "6_11_19", "6_10_19", "6_8_20", "6_5_11", "6_5_8", "6_9_4", "6_2_21", "6_11_1", "6_6_13", "6_10_15", "6_2_7", "6_8_14", "6_3_3", "6_9_10", "6_5_0", "6_6_5", "6_3_0", "6_10_4", "6_6_23", "6_12_12", "6_3_15", "6_8_11", "6_10_21", "6_3_10", "6_13_5", "6_6_15", "6_12_6", "6_10_9", "6_11_23", "6_5_4", "6_6_7", "6_8_21", "6_12_21", "6_8_18", "7_12_6", "7_8_1", "7_11_17", "7_2_3", "7_10_14", "7_6_9", "7_5_9", "7_6_23", "7_5_14", "7_3_13", "7_2_5", "7_13_20", "7_10_24", "7_13_21", "7_5_20", "7_8_20", "7_12_18", "7_8_18", "7_11_16", "7_3_17", "7_10_21", "7_5_13", "7_2_13", "7_11_3", "7_5_4", "7_2_20", "7_9_10", "7_6_24", "7_13_15", "7_2_21", "7_2_4", "7_8_7", "7_13_9", "7_8_4", "7_5_5", "7_13_8", "7_6_2", "7_13_10", "7_9_13", "7_8_23", "7_9_11", "7_8_11", "7_9_19", "7_3_12", "7_10_6", "7_8_10", "7_3_3", "7_11_15", "7_3_7", "7_6_14", "7_2_9", "7_11_13", "7_12_7", "7_3_10", "7_5_8", "7_2_24", "7_10_12", "7_11_7", "7_11_10", "7_13_16", "7_10_1", "7_6_15", "7_8_5", "7_3_20", "7_6_0", "7_9_8", "7_12_17", "7_12_24", "7_2_0", "7_10_20", "7_10_2", "7_9_1", "7_11_5", "7_12_16", "7_11_12", "7_9_9", "7_3_2", "7_6_21", "7_13_5", "7_10_23", "7_11_14", "7_9_3", "7_2_15", "7_6_7", "7_2_17", "7_6_3", "7_5_22", "7_9_21", "7_12_23", "7_10_18", "7_3_14", "7_8_2", "7_5_23", "7_12_5", "7_13_23", "7_2_16", "7_12_22", "7_9_4", "7_13_17", "7_6_4", "7_10_11", "7_10_7", "7_3_1", "7_5_15", "7_9_14", "7_2_1", "7_9_12", "7_5_11", "7_11_9", "7_6_20", "7_6_19", "7_12_0", "7_13_2", "7_6_1", "7_13_11", "7_11_21", "7_8_19", "7_11_23", "7_8_13", "7_12_20", "7_2_22", "7_11_22", "7_10_22", "7_9_0", "7_3_19", "7_3_5", "8_11_7", "8_3_20", "8_13_6", "8_2_5", "8_13_16", "8_13_2", "8_10_14", "8_13_20", "8_8_0", "8_2_20", "8_2_24", "8_2_7", "8_11_2", "8_2_12", "8_2_14", "8_2_17", "8_10_11", "8_6_22", "8_10_19", "8_12_17", "8_2_11", "8_9_9", "8_9_0", "8_11_11", "8_13_0", "8_11_6", "8_5_24", "8_10_5", "8_5_9", "8_13_4", "8_3_2", "8_6_16", "8_13_5", "8_8_11", "8_5_19", "8_10_7", "8_12_13", "8_13_14", "8_8_3", "8_12_18", "8_3_16", "8_12_23", "8_10_8", "8_11_17", "8_8_16", "8_3_9", "8_8_22", "8_11_3", "8_10_24", "8_3_23", "8_6_13", "8_5_17", "8_2_4", "8_13_21", "8_12_1", "8_10_17", "8_6_2", "8_6_7", "8_11_24", "8_3_10", "8_12_20", "8_3_3", "8_12_14", "8_5_6", "8_8_18", "8_11_13", "8_2_23", "8_2_1", "8_12_12", "8_8_6", "8_12_16", "8_5_4", "8_3_6", "8_12_2", "8_10_3", "8_5_23", "8_12_4", "8_11_12", "8_6_5", "8_13_24", "8_3_19", "8_10_18", "8_12_21", "8_9_16", "8_9_3", "8_3_22", "8_8_9", "8_3_17", "8_9_7", "8_5_22", "8_8_10", "8_9_11", "8_12_3", "8_9_22", "8_2_13", "8_6_23", "8_6_20", "8_6_18", "8_12_7", "8_10_0", "8_8_7", "8_12_19", "8_8_20", "8_12_9", "8_9_14", "8_12_15", "8_3_4", "8_11_1", "8_6_14", "8_6_15", "8_9_13", "8_10_10", "8_3_12", "8_5_12", "8_2_21", "8_13_7", "8_10_20", "8_6_3", "8_8_8", "8_6_12", "8_5_3", "8_6_8", "8_11_0", "8_8_19", "8_11_19", "8_9_5", "9_8_3", "9_9_18", "9_8_13", "9_9_16", "9_8_5", "9_11_23", "9_6_1", "9_9_17", "9_3_20", "9_2_23", "9_9_10", "9_12_21", "9_5_21", "9_10_10", "9_8_22", "9_6_7", "9_3_23", "9_9_21", "9_11_8", "9_2_17", "9_9_12", "9_11_15", "9_11_6", "9_2_21", "9_2_0", "9_13_1", "9_9_20", "9_5_22", "9_12_20", "9_2_14", "9_10_23", "9_5_7", "9_12_0", "9_3_11", "9_10_5", "9_2_16", "9_5_12", "9_8_16", "9_13_9", "9_12_15", "9_12_24", "9_5_14", "9_9_13", "9_13_0", "9_6_4", "9_9_23", "9_13_19", "9_13_12", "9_9_4", "9_9_5", "9_8_12", "9_13_23", "9_5_9", "9_8_14", "9_13_4", "9_3_10", "9_5_11", "9_3_9", "9_3_3", "9_11_14", "9_2_22", "9_9_15", "9_13_3", "9_10_19", "9_11_12", "9_12_22", "9_8_1", "9_11_5", "9_10_16", "9_3_21", "9_10_18", "9_12_17", "9_10_15", "9_3_7", "9_5_18", "9_5_19", "9_2_4", "9_12_3", "9_3_17", "9_2_5", "9_13_24", "9_6_17", "9_6_11", "9_11_21", "9_10_0", "9_11_18", "9_10_11", "9_8_24", "9_12_18", "9_10_2", "9_2_8", "9_9_3", "9_11_9", "9_5_23", "9_5_15", "9_12_6", "9_9_22", "9_2_10", "9_2_3", "9_10_12", "9_11_17", "9_3_14", "9_13_14", "9_8_0", "9_3_8", "9_5_1", "9_13_17", "9_12_12", "9_6_19", "9_3_15", "9_8_11", "9_3_4", "9_10_21", "9_6_21", "9_10_9", "9_8_2", "9_6_10", "9_8_4", "9_2_13", "9_3_1", "9_6_20", "9_10_14", "9_8_7", "9_3_13", "9_5_0", "9_5_17", "10_10_16", "10_10_7", "10_3_23", "10_5_0", "10_6_16", "10_10_11", "10_6_2", "10_5_9", "10_12_7", "10_10_19", "10_13_15", "10_10_3", "10_6_7", "10_11_17", "10_9_0", "10_6_5", "10_3_11", "10_13_0", "10_13_2", "10_10_13", "10_11_20", "10_3_19", "10_13_14", "10_5_15", "10_9_19", "10_9_22", "10_11_3", "10_2_1", "10_6_13", "10_12_11", "10_9_20", "10_8_6", "10_10_20", "10_8_2", "10_8_19", "10_9_14", "10_2_14", "10_13_18", "10_9_16", "10_2_13", "10_9_7", "10_3_1", "10_9_4", "10_2_15", "10_9_6", "10_8_4", "10_12_13", "10_13_10", "10_12_1", "10_12_2", "10_13_13", "10_8_0", "10_10_6", "10_12_20", "10_5_16", "10_2_20", "10_6_8", "10_12_15", "10_2_2", "10_9_1", "10_13_19", "10_9_15", "10_8_20", "10_8_22", "10_2_8", "10_2_7", "10_11_23", "10_10_1", "10_9_17", "10_10_5", "10_10_18", "10_11_8", "10_6_17", "10_11_12", "10_3_15", "10_13_20", "10_3_17", "10_11_0", "10_2_24", "10_12_12", "10_12_10", "10_13_4", "10_8_16", "10_8_11", "10_8_3", "10_6_18", "10_8_10", "10_5_14", "10_12_17", "10_6_23", "10_3_13", "10_13_12", "10_5_19", "10_6_20", "10_11_21", "10_11_7", "10_12_4", "10_13_5", "10_2_12", "10_11_6", "10_10_15", "10_2_19", "10_10_22", "10_13_9", "10_5_5", "10_5_1", "10_5_24", "10_2_9", "10_6_22", "10_12_16", "10_5_6", "10_13_11", "10_13_21", "10_10_0", "10_9_10", "10_10_8", "10_9_3", "10_5_12", "10_12_19", "10_8_9", "10_3_12", "10_11_11", "10_11_16", "10_5_4", "10_6_19", "10_3_7"], "eval": ["0_12_20", "0_10_18", "0_6_8", "0_3_0", "0_12_22", "0_13_10", "0_12_3", "0_5_14", "0_10_15", "0_2_5", "0_3_5", "0_3_18", "0_12_21", "0_5_21", "0_10_20", "0_8_22", "0_8_11", "0_13_6", "0_9_0", "0_3_7", "0_8_8", "0_5_10", "0_10_6", "0_9_20", "0_6_20", "0_13_15", "0_6_2", "0_9_13", "0_13_17", "0_12_23", "0_2_4", "0_12_10", "0_11_12", "0_13_21", "0_13_9", "0_12_6", "0_5_12", "0_9_16", "0_2_18", "0_13_4", "0_6_1", "0_10_12", "0_11_1", "0_11_6", "0_13_16", "0_2_13", "0_2_12", "0_5_16", "0_8_10", "0_13_19", "0_12_2", "0_12_17", "0_9_19", "0_6_3", "0_13_2", "0_9_6", "0_5_13", "0_12_9", "0_6_21", "0_5_19", "0_10_9", "0_9_23", "0_2_15", "0_9_22", "0_3_15", "0_11_21", "0_8_18", "0_12_12", "0_12_8", "0_5_22", "0_10_23", "0_5_17", "0_13_24", "0_6_12", "0_6_13", "0_10_24", "0_8_0", "0_5_24", "0_12_0", "0_8_6", "0_8_15", "0_5_11", "0_2_20", "0_8_2", "0_6_16", "0_9_21", "0_6_14", "0_3_21", "0_10_4", "0_13_8", "0_10_22", "0_9_18", "0_6_24", "0_13_1", "0_11_10", "0_5_4", "0_8_1", "0_13_11", "0_10_19", "0_2_23", "0_13_3", "0_11_8", "0_8_9", "0_12_15", "0_11_15", "0_3_24", "0_8_17", "0_3_10", "0_5_1", "0_2_21", "0_10_17", "0_2_8", "0_9_8", "0_13_7", "0_6_9", "0_5_20", "0_12_24", "0_12_11", "0_13_0", "0_9_11", "0_5_23", "0_11_11", "0_6_23", "0_13_14", "1_6_23", "1_9_5", "1_11_16", "1_10_16", "1_13_22", "1_3_18", "1_6_19", "1_8_23", "1_9_20", "1_8_9", "1_8_24", "1_9_15", "1_12_9", "1_5_2", "1_5_24", "1_6_17", "1_10_3", "1_10_22", "1_6_7", "1_12_1", "1_9_10", "1_12_10", "1_6_5", "1_8_21", "1_5_17", "1_3_20", "1_13_23", "1_2_15", "1_8_7", "1_11_1", "1_2_7", "1_11_23", "1_13_8", "1_6_13", "1_2_21", "1_3_13", "1_13_15", "1_6_18", "1_9_9", "1_2_19", "1_12_17", "1_11_11", "1_5_20", "1_5_16", "1_12_6", "1_3_7", "1_2_20", "1_11_7", "1_6_1", "1_9_22", "1_10_0", "1_6_22", "1_9_14", "1_2_3", "1_6_6", "1_8_17", "1_3_19", "1_11_0", "1_8_18", "1_10_15", "1_9_8", "1_9_18", "1_5_0", "1_10_23", "1_5_23", "1_10_13", "1_10_7", "1_6_8", "1_11_6", "1_8_0", "1_8_6", "1_13_17", "1_12_4", "1_13_13", "1_9_0", "1_6_20", "1_5_4", "1_3_3", "1_3_23", "1_12_20", "1_10_4", "1_11_2", "1_8_20", "1_12_3", "1_8_8", "1_12_8", "1_12_0", "1_5_11", "1_13_12", "1_9_11", "1_2_1", "1_12_13", "1_10_17", "1_3_22", "1_6_9", "1_3_12", "1_12_16", "1_8_15", "1_12_23", "1_5_5", "1_11_5", "1_11_20", "1_13_16", "1_8_4", "1_5_13", "1_13_18", "1_8_13", "1_8_1", "1_10_9", "1_13_4", "1_6_12", "1_5_7", "1_6_21", "1_9_2", "1_3_11", "1_5_3", "1_10_19", "1_9_4", "1_9_3", "1_13_0", "1_12_19", "1_10_11", "1_11_13", "1_12_21", "2_6_24", "2_10_13", "2_12_18", "2_5_2", "2_2_10", "2_12_4", "2_2_15", "2_10_14", "2_9_11", "2_9_19", "2_8_5", "2_9_14", "2_10_6", "2_3_18", "2_13_12", "2_2_0", "2_11_19", "2_12_21", "2_5_0", "2_8_19", "2_9_18", "2_13_5", "2_3_12", "2_3_6", "2_12_6", "2_9_21", "2_9_4", "2_3_14", "2_11_10", "2_11_1", "2_9_6", "2_2_17", "2_5_16", "2_6_9", "2_2_16", "2_6_11", "2_9_9", "2_9_12", "2_8_15", "2_2_1", "2_6_10", "2_13_16", "2_12_1", "2_9_13", "2_10_3", "2_6_4", "2_6_1", "2_11_12", "2_12_12", "2_11_7", "2_3_17", "2_6_17", "2_9_0", "2_2_8", "2_3_24", "2_9_3", "2_8_6", "2_13_0", "2_9_2", "2_5_10", "2_13_4", "2_3_10", "2_6_3", "2_9_1", "2_3_11", "2_9_10", "2_13_14", "2_5_7", "2_6_19", "2_13_19", "2_10_1", "2_2_19", "2_11_20", "2_13_24", "2_10_4", "2_6_18", "2_9_15", "2_6_15", "2_5_22", "2_13_3", "2_10_16", "2_11_11", "2_8_17", "2_10_2", "2_6_0", "2_3_21", "2_3_15", "2_13_8", "2_6_22", "2_10_9", "2_12_16", "2_5_18", "2_8_10", "2_10_15", "2_8_22", "2_13_13", "2_12_8", "2_13_15", "2_11_22", "2_2_4", "2_11_13", "2_3_4", "2_5_4", "2_11_4", "2_8_14", "2_2_6", "2_9_16", "2_11_23", "2_12_13", "2_3_9", "2_8_20", "2_6_20", "2_11_15", "2_6_21", "2_13_22", "2_5_9", "2_10_8", "2_12_0", "2_11_3", "2_2_24", "2_5_24", "2_13_23", "2_8_8", "2_2_21", "3_13_24", "3_2_17", "3_11_3", "3_5_9", "3_2_0", "3_3_23", "3_10_23", "3_3_6", "3_10_9", "3_8_18", "3_5_12", "3_11_17", "3_8_5", "3_2_15", "3_6_14", "3_9_12", "3_9_9", "3_13_10", "3_11_14", "3_8_3", "3_2_8", "3_11_8", "3_8_1", "3_3_15", "3_5_0", "3_13_0", "3_8_2", "3_6_7", "3_13_3", "3_11_1", "3_2_20", "3_8_16", "3_5_3", "3_2_13", "3_2_7", "3_2_4", "3_5_18", "3_6_3", "3_3_7", "3_9_18", "3_6_15", "3_9_15", "3_2_24", "3_11_22", "3_2_18", "3_10_12", "3_8_0", "3_8_13", "3_8_21", "3_9_6", "3_3_21", "3_12_13", "3_3_1", "3_8_15", "3_11_4", "3_3_0", "3_13_19", "3_11_16", "3_6_9", "3_2_19", "3_9_17", "3_13_11", "3_5_20", "3_13_7", "3_3_3", "3_13_23", "3_9_16", "3_9_8", "3_5_13", "3_5_23", "3_10_15", "3_8_19", "3_12_5", "3_12_4", "3_9_19", "3_6_20", "3_11_21", "3_6_1", "3_11_12", "3_2_22", "3_2_5", "3_6_13", "3_12_6", "3_2_16", "3_9_0", "3_8_23", "3_9_10", "3_6_0", "3_11_15", "3_12_9", "3_5_1", "3_13_20", "3_13_8", "3_13_6", "3_10_5", "3_10_24", "3_8_22", "3_6_10", "3_8_6", "3_8_7", "3_9_5", "3_11_11", "3_2_14", "3_6_2", "3_5_10", "3_11_23", "3_8_11", "3_10_0", "3_2_1", "3_11_24", "3_12_23", "3_12_17", "3_12_0", "3_12_24", "3_12_2", "3_9_2", "3_3_9", "3_5_7", "3_13_16", "3_10_4", "3_12_8", "3_6_4", "3_2_6", "3_3_24", "4_12_3", "4_6_12", "4_10_13", "4_3_21", "4_6_21", "4_3_15", "4_11_16", "4_13_5", "4_3_4", "4_10_2", "4_6_7", "4_9_11", "4_10_8", "4_2_20", "4_3_6", "4_3_2", "4_11_11", "4_8_16", "4_2_0", "4_2_2", "4_5_6", "4_12_1", "4_6_19", "4_8_0", "4_12_8", "4_13_10", "4_2_21", "4_3_24", "4_12_22", "4_11_8", "4_5_17", "4_13_4", "4_8_1", "4_11_14", "4_11_20", "4_10_21", "4_13_2", "4_13_1", "4_8_12", "4_9_0", "4_12_11", "4_12_2", "4_2_24", "4_3_11", "4_6_16", "4_12_7", "4_6_3", "4_9_13", "4_9_2", "4_13_6", "4_6_9", "4_8_18", "4_12_24", "4_2_11", "4_2_18", "4_9_8", "4_5_22", "4_5_10", "4_11_24", "4_13_22", "4_13_17", "4_8_11", "4_13_14", "4_8_20", "4_11_13", "4_2_1", "4_12_16", "4_11_15", "4_13_0", "4_2_5", "4_12_20", "4_5_5", "4_10_12", "4_8_21", "4_13_13", "4_10_6", "4_9_7", "4_2_19", "4_10_15", "4_10_22", "4_11_1", "4_10_7", "4_12_0", "4_2_12", "4_12_15", "4_9_9", "4_8_17", "4_8_8", "4_12_10", "4_3_23", "4_6_22", "4_8_4", "4_11_10", "4_2_9", "4_5_23", "4_6_4", "4_5_3", "4_13_12", "4_3_10", "4_10_4", "4_12_6", "4_6_13", "4_10_10", "4_9_12", "4_8_6", "4_13_16", "4_2_17", "4_2_10", "4_13_8", "4_11_21", "4_12_12", "4_11_9", "4_8_3", "4_13_18", "4_5_9", "4_5_20", "4_10_14", "4_6_5", "4_10_5", "4_11_2", "4_6_11", "4_5_13", "4_5_21", "4_6_17", "5_8_21", "5_10_19", "5_2_8", "5_5_16", "5_9_21", "5_2_20", "5_6_18", "5_6_8", "5_9_18", "5_12_11", "5_10_17", "5_13_11", "5_8_3", "5_5_13", "5_11_7", "5_12_8", "5_2_0", "5_3_23", "5_8_17", "5_5_6", "5_10_5", "5_8_9", "5_10_21", "5_12_10", "5_3_16", "5_5_14", "5_9_1", "5_11_21", "5_11_16", "5_5_22", "5_2_24", "5_13_15", "5_8_0", "5_10_1", "5_8_14", "5_8_20", "5_12_19", "5_8_6", "5_6_7", "5_5_1", "5_11_14", "5_3_19", "5_11_9", "5_5_24", "5_6_20", "5_8_24", "5_3_12", "5_12_2", "5_5_2", "5_12_16", "5_3_0", "5_11_10", "5_11_20", "5_13_19", "5_5_5", "5_3_15", "5_9_14", "5_9_24", "5_13_24", "5_11_23", "5_13_7", "5_13_23", "5_6_9", "5_6_24", "5_2_12", "5_5_7", "5_3_22", "5_12_12", "5_5_18", "5_12_23", "5_13_22", "5_2_23", "5_11_24", "5_11_17", "5_2_14", "5_3_4", "5_11_6", "5_11_0", "5_6_6", "5_10_4", "5_5_20", "5_11_15", "5_10_3", "5_3_10", "5_11_2", "5_9_4", "5_9_2", "5_3_14", "5_10_7", "5_10_23", "5_11_3", "5_6_4", "5_9_19", "5_9_8", "5_6_22", "5_8_7", "5_6_11", "5_9_9", "5_6_23", "5_9_15", "5_3_1", "5_12_20", "5_5_3", "5_12_21", "5_2_22", "5_13_16", "5_8_15", "5_10_18", "5_13_20", "5_9_12", "5_6_10", "5_10_10", "5_13_10", "5_8_2", "5_13_17", "5_10_12", "5_12_13", "5_3_9", "5_6_12", "5_2_16", "5_11_19", "5_2_5", "5_2_1", "5_11_22", "6_2_11", "6_2_17", "6_11_13", "6_2_24", "6_13_18", "6_5_19", "6_6_14", "6_12_1", "6_13_23", "6_5_15", "6_10_6", "6_12_19", "6_12_22", "6_6_12", "6_6_1", "6_10_17", "6_3_23", "6_13_9", "6_9_5", "6_13_19", "6_6_4", "6_5_22", "6_12_15", "6_5_21", "6_8_1", "6_12_7", "6_2_9", "6_11_16", "6_2_5", "6_2_18", "6_5_7", "6_6_6", "6_11_7", "6_11_6", "6_12_14", "6_11_9", "6_12_20", "6_8_16", "6_10_18", "6_2_19", "6_10_2", "6_2_22", "6_3_22", "6_6_18", "6_11_22", "6_13_22", "6_5_24", "6_3_19", "6_12_5", "6_5_17", "6_8_7", "6_8_19", "6_8_6", "6_13_8", "6_13_2", "6_9_19", "6_11_17", "6_11_4", "6_9_7", "6_13_21", "6_5_6", "6_11_0", "6_5_18", "6_2_10", "6_13_3", "6_13_24", "6_8_10", "6_12_0", "6_10_24", "6_13_11", "6_5_23", "6_2_3", "6_13_16", "6_13_6", "6_3_12", "6_12_17", "6_11_15", "6_11_24", "6_12_2", "6_3_21", "6_12_18", "6_9_23", "6_8_24", "6_2_13", "6_6_17", "6_6_0", "6_6_3", "6_6_8", "6_13_15", "6_3_8", "6_11_10", "6_10_16", "6_8_22", "6_2_2", "6_5_9", "6_8_5", "6_2_23", "6_11_20", "6_10_11", "6_11_14", "6_2_6", "6_2_0", "6_3_1", "6_3_17", "6_13_4", "6_13_10", "6_10_22", "6_11_8", "6_5_16", "6_8_4", "6_13_17", "6_2_14", "6_10_23", "6_5_3", "6_11_5", "6_12_8", "6_11_12", "6_13_14", "6_9_17", "6_11_18", "6_12_3", "6_12_13", "6_3_16", "6_3_7", "7_6_5", "7_12_19", "7_11_18", "7_2_6", "7_9_23", "7_3_21", "7_3_23", "7_12_2", "7_12_14", "7_6_16", "7_12_3", "7_8_9", "7_10_4", "7_2_12", "7_12_8", "7_5_12", "7_9_15", "7_9_16", "7_6_11", "7_9_5", "7_11_4", "7_8_0", "7_5_21", "7_5_19", "7_5_10", "7_9_7", "7_2_7", "7_8_22", "7_9_24", "7_10_5", "7_10_15", "7_5_7", "7_10_13", "7_6_17", "7_2_23", "7_9_20", "7_11_20", "7_13_14", "7_12_12", "7_9_2", "7_3_22", "7_3_18", "7_5_3", "7_9_6", "7_11_0", "7_8_15", "7_6_6", "7_5_1", "7_13_24", "7_3_4", "7_9_17", "7_5_2", "7_12_1", "7_13_0", "7_9_22", "7_12_9", "7_5_17", "7_12_10", "7_13_3", "7_2_10", "7_6_8", "7_2_18", "7_6_18", "7_13_13", "7_11_1", "7_12_21", "7_12_15", "7_13_18", "7_8_21", "7_13_7", "7_3_9", "7_3_8", "7_2_11", "7_8_12", "7_2_2", "7_12_4", "7_6_12", "7_10_16", "7_13_4", "7_5_16", "7_10_9", "7_11_19", "7_3_16", "7_3_15", "7_2_8", "7_11_2", "7_3_24", "7_6_13", "7_10_19", "7_10_0", "7_8_6", "7_10_17", "7_10_3", "7_8_14", "7_13_22", "7_2_19", "7_8_8", "7_3_11", "7_3_6", "7_5_24", "7_5_0", "7_3_0", "7_8_17", "7_12_11", "7_5_18", "7_11_6", "7_8_3", "7_6_22", "7_11_11", "7_9_18", "7_5_6", "7_13_19", "7_10_8", "7_11_24", "7_12_13", "7_8_24", "7_13_6", "7_13_12", "7_8_16", "7_10_10", "7_13_1", "7_6_10", "7_11_8", "7_2_14", "8_11_15", "8_12_10", "8_13_11", "8_10_6", "8_6_21", "8_5_10", "8_9_10", "8_13_17", "8_10_22", "8_9_20", "8_8_12", "8_8_2", "8_5_2", "8_11_23", "8_13_22", "8_3_21", "8_12_8", "8_2_16", "8_3_18", "8_8_17", "8_9_15", "8_2_10", "8_5_1", "8_9_23", "8_9_2", "8_8_1", "8_6_10", "8_5_11", "8_2_3", "8_5_7", "8_2_6", "8_11_8", "8_8_4", "8_6_9", "8_5_5", "8_9_8", "8_10_1", "8_13_1", "8_9_1", "8_3_0", "8_3_13", "8_11_21", "8_8_5", "8_13_13", "8_6_24", "8_6_11", "8_2_9", "8_5_15", "8_5_13", "8_2_8", "8_13_15", "8_10_12", "8_11_10", "8_10_2", "8_5_0", "8_9_24", "8_9_21", "8_3_15", "8_2_19", "8_2_22", "8_10_13", "8_5_8", "8_5_16", "8_9_6", "8_9_19", "8_6_17", "8_3_7", "8_11_5", "8_6_19", "8_12_0", "8_2_2", "8_2_0", "8_10_4", "8_8_23", "8_10_16", "8_11_18", "8_13_3", "8_11_9", "8_3_14", "8_3_5", "8_13_8", "8_5_18", "8_10_23", "8_2_18", "8_13_23", "8_9_18", "8_13_12", "8_8_14", "8_12_24", "8_12_5", "8_9_12", "8_3_1", "8_11_16", "8_6_6", "8_12_11", "8_8_15", "8_8_24", "8_6_4", "8_2_15", "8_11_20", "8_11_14", "8_13_19", "8_10_15", "8_10_9", "8_13_10", "8_12_22", "8_13_9", "8_11_22", "8_5_20", "8_11_4", "8_3_24", "8_8_13", "8_9_17", "8_6_0", "8_12_6", "8_3_11", "8_6_1", "8_9_4", "8_13_18", "8_5_14", "8_10_21", "8_8_21", "8_5_21", "8_3_8", "9_2_1", "9_9_14", "9_10_1", "9_9_1", "9_13_2", "9_3_0", "9_11_24", "9_11_0", "9_12_23", "9_2_24", "9_6_8", "9_11_3", "9_6_9", "9_2_18", "9_6_24", "9_11_16", "9_2_12", "9_2_15", "9_2_20", "9_13_13", "9_13_6", "9_6_14", "9_2_9", "9_2_11", "9_6_22", "9_3_16", "9_9_2", "9_9_19", "9_13_16", "9_9_8", "9_12_4", "9_13_7", "9_12_7", "9_13_10", "9_3_19", "9_8_10", "9_10_13", "9_13_15", "9_8_19", "9_12_1", "9_11_4", "9_12_10", "9_13_5", "9_10_3", "9_12_19", "9_3_18", "9_5_16", "9_3_22", "9_11_22", "9_5_8", "9_5_20", "9_13_22", "9_10_17", "9_12_16", "9_10_24", "9_13_8", "9_8_15", "9_13_18", "9_3_24", "9_2_2", "9_8_20", "9_8_17", "9_10_20", "9_11_2", "9_9_6", "9_8_9", "9_8_23", "9_5_10", "9_8_18", "9_12_9", "9_10_7", "9_3_2", "9_13_11", "9_6_3", "9_9_24", "9_6_18", "9_6_0", "9_10_8", "9_9_9", "9_6_6", "9_11_7", "9_6_5", "9_11_13", "9_13_20", "9_11_10", "9_9_11", "9_10_22", "9_11_19", "9_6_16", "9_12_13", "9_6_12", "9_12_5", "9_9_7", "9_5_6", "9_5_5", "9_12_14", "9_9_0", "9_6_23", "9_10_4", "9_3_5", "9_6_2", "9_3_6", "9_6_13", "9_11_11", "9_5_13", "9_2_19", "9_12_11", "9_11_20", "9_8_21", "9_11_1", "9_5_4", "9_2_6", "9_5_3", "9_5_2", "9_8_8", "9_6_15", "9_2_7", "9_5_24", "9_12_8", "9_13_21", "9_3_12", "9_8_6", "9_10_6", "9_12_2", "10_13_24", "10_13_16", "10_12_8", "10_3_2", "10_12_18", "10_12_3", "10_9_2", "10_10_4", "10_9_9", "10_9_5", "10_8_18", "10_2_6", "10_13_1", "10_9_21", "10_11_1", "10_10_17", "10_13_17", "10_2_17", "10_13_6", "10_12_24", "10_2_5", "10_9_11", "10_13_7", "10_9_8", "10_10_2", "10_5_8", "10_12_23", "10_6_11", "10_3_24", "10_5_3", "10_6_15", "10_11_22", "10_9_24", "10_10_23", "10_9_18", "10_8_23", "10_6_6", "10_2_21", "10_10_21", "10_8_12", "10_2_3", "10_8_7", "10_6_9", "10_8_14", "10_5_17", "10_3_16", "10_11_5", "10_8_8", "10_3_21", "10_3_18", "10_3_10", "10_2_10", "10_8_15", "10_8_5", "10_6_4", "10_10_14", "10_2_4", "10_3_0", "10_11_18", "10_9_12", "10_6_24", "10_12_5", "10_13_3", "10_3_8", "10_3_9", "10_12_0", "10_6_10", "10_2_23", "10_9_23", "10_11_10", "10_11_9", "10_11_14", "10_2_16", "10_13_8", "10_3_22", "10_5_13", "10_5_22", "10_12_9", "10_3_6", "10_5_21", "10_5_2", "10_10_9", "10_11_19", "10_5_20", "10_3_14", "10_5_10", "10_11_15", "10_10_10", "10_8_21", "10_12_22", "10_11_24", "10_6_14", "10_12_14", "10_5_7", "10_11_13", "10_8_24", "10_6_12", "10_6_21", "10_6_1", "10_5_23", "10_6_0", "10_5_18", "10_11_4", "10_12_21", "10_3_3", "10_2_11", "10_11_2", "10_5_11", "10_6_3", "10_10_24", "10_9_13", "10_8_13", "10_8_17", "10_3_20", "10_13_22", "10_2_22", "10_13_23", "10_8_1", "10_10_12", "10_2_18", "10_3_4", "10_2_0", "10_12_6", "10_3_5"]}
================================================
FILE: image/Makefile
================================================
CC=c++
# c flags
CFLAGS=-c -Wall -std=c++11 -O3
CFLAGS_BOOST=-DBOOST_SYSTEM_NO_DEPRECATED
CFLAGS_CV=`pkg-config --cflags opencv`
CFLAGS_H5=-I/usr/include -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE \
-D_BSD_SOURCE -D_FORTIFY_SOURCE=2 -g -fstack-protector \
--param=ssp-buffer-size=4 -Wformat -Werror=format-security
# linking flags
LDFLAGS_BOOST=-lboost_system -lboost_program_options
LDFLAGS_H5=-lhdf5 -lhdf5_cpp
LDFLAGS_CV=-L/usr/local/cuda-7.5/lib64 `pkg-config --libs opencv`
BIN=bin
SOURCES=$(wildcard *.cpp)
OBJECTS=$(patsubst %.cpp, %.o, $(SOURCES))
EXECUTABLE=$(BIN)/image
all: $(SOURCES) $(EXECUTABLE)
$(BIN)/image: $(BIN)/main.o $(BIN)/h5Data.o $(BIN)/seq.o
$(CC) $(BIN)/main.o $(BIN)/h5Data.o $(BIN)/seq.o \
$(LDFLAGS_BOOST) $(LDFLAGS_H5) $(LDFLAGS_CV) -o $@
$(BIN)/main.o: main.cpp
$(CC) $(CFLAGS) $(CFLAGS_BOOST) $< -o $@
$(BIN)/h5Data.o: h5Data.cpp
$(CC) $(CFLAGS) $(CFLAGS_H5) $< -o $@
$(BIN)/seq.o: seq.cpp
$(CC) $(CFLAGS) $(CFLAGS_CV) $< -o $@
clean:
rm -r -f $(BIN)/*
================================================
FILE: image/h5Data.cpp
================================================
#include "h5Data.hpp"
h5Data::h5Data(std::string f, int n) : fileName(f), numChannel(n) {
// turn off the auto-printing when failure occurs
dataFile = new H5::H5File(fileName, H5F_ACC_RDONLY);
datasetLa = load(datasetNameLa, -1);
for (int i = 0; i < numChannel; i++) {
datasetCh[i] = load(datasetNameCh + std::to_string(i), i);
}
}
h5Data::~h5Data() {
// delete array of label and channels with []
delete[] label;
delete datasetLa;
for (int i = 0; i < numChannel; i++) {
delete[] ch[i];
delete datasetCh[i];
}
delete dataFile;
}
H5::DataSet* h5Data::load(H5std_string datasetName, int whichCh) {
// open dataset, dataspace and property list
try {
H5::Exception::dontPrint();
hsize_t dimsr[2];
H5::DataSet* dataset = new H5::DataSet(dataFile->openDataSet(datasetName));
H5::DataSpace *filespace = new H5::DataSpace(dataset->getSpace());
H5::DSetCreatPropList prop = dataset->getCreatePlist();
// get information to obtain memory dataspace.
int rank = filespace->getSimpleExtentNdims();
filespace->getSimpleExtentDims(dimsr);
H5::DataSpace* memspace = new H5::DataSpace(rank, dimsr, NULL);
length = dimsr[0];
// load labels and channels
if (whichCh == -1) {
label = new int[length];
dataset->read(label, H5::PredType::NATIVE_INT, *memspace, *filespace);
} else {
ch[whichCh] = new float[length * dimsr[1]];
dataset->read(ch[whichCh], H5::PredType::NATIVE_FLOAT, *memspace,
*filespace);
}
prop.close();
delete filespace;
delete memspace;
return dataset;
} catch (H5::FileIException error) {
error.printError();
return NULL;
} catch (H5::DataSetIException error) {
error.printError();
return NULL;
} catch (H5::DataSpaceIException error) {
error.printError();
return NULL;
}
}
================================================
FILE: image/h5Data.hpp
================================================
#ifndef H5DATA_H
#define H5DATA_H
#include
#include
// hdf5 includes
#include "H5Cpp.h"
// h5 tags
namespace {
const H5std_string datasetNameLa = "label";
const H5std_string datasetNameCh = "ch";
const hsize_t dimsLa[2] = {1, 1};
const hsize_t dimsCh[2] = {1, 32 * 32};
const int backgroundClass = 0;
}
#endif
// data class for 5f file
class h5Data {
private:
std::string fileName;
int numChannel;
int length;
// store dataset
H5::H5File *dataFile = NULL;
H5::DataSet *datasetLa = NULL;
H5::DataSet *datasetCh[8];
// store label and channels data
int *label;
float *ch[8];
public:
h5Data(std::string f, int n);
~h5Data();
H5::DataSet* load(H5std_string datasetName, int whichCh);
inline float* getCh(int whichCh) { return ch[whichCh]; }
inline int* getLa() { return label; }
inline int getLength() { return length; }
};
================================================
FILE: image/main.cpp
================================================
#include
#include
// boost options
#include
namespace po = boost::program_options;
#include "seq.hpp"
int main(int argc, char *argv[]) {
try {
// generate argument parser
po::options_description desc("Image: allowed options");
desc.add_options()
("help", "Print help messages")
("source", po::value(), "h5 source file")
("target", po::value(), "Flow target directory")
("channel", po::value(), "Channel number")
("origin", po::value(), "Origin size")
("out", po::value(), "Output size")
("label", po::value(), "Print label in output file name")
("bound", po::value(), "Bound for normalization");
po::variables_map vm;
po::store(po::parse_command_line(argc, argv, desc), vm);
po::notify(vm);
std::string source, target;
int numChannel = 1, origin = 32, out = 32;
bool label = false;
double bound = 15.0;
// assign argument to local variables
if (vm.count("help")) {
std::cout << "Image" << std::endl << desc << std::endl;
return 0;
} else {
if (vm.count("source")) source = vm["source"].as();
if (vm.count("target")) target = vm["target"].as();
if (vm.count("channel")) numChannel = vm["channel"].as();
if (vm.count("origin")) origin = vm["origin"].as();
if (vm.count("out")) out = vm["out"].as();
if (vm.count("label")) label = vm["label"].as();
if (vm.count("bound")) bound = vm["bound"].as();
// generate flow for different channels
Seq seq = Seq(source, target, numChannel, label);
for (int i = 0; i < numChannel; i++) {
seq.flow(i, origin, origin, out, out, bound);
}
}
} catch (std::exception& e) {
std::cerr << "Error: " << e.what() << std::endl;
return 1;
} catch (...) {
std::cerr << "Exception of unknown type" << std::endl;
return 1;
}
return 0;
}
================================================
FILE: image/seq.cpp
================================================
#include "seq.hpp"
Seq::Seq(std::string s, std::string t, int n, bool l) :
source(s), target(t), numChannel(n), printLabel(l) {
// read h5 file and get the data
h5file = new h5Data(source, numChannel);
label = h5file->getLa();
for (int i = 0; i < numChannel; i++) {
ch[i] = h5file->getCh(i);
}
length = h5file->getLength();
}
Seq::~Seq() {
// delete files with pointer at the end, release channel data in the end
delete h5file;
}
// convert float (0-1) value images to int (0-255) images with bound limit
void Seq::float2Image(const cv::Mat &floatMat, cv::Mat &imageMat,
double l, double u) {
for (int i = 0; i < floatMat.rows; i++) {
for (int j = 0; j < floatMat.cols; j++) {
float x = floatMat.at(i, j);
imageMat.at(i, j) = cast(x, l, u);
}
}
}
// load array data image with offset to int images with resize
cv::Mat Seq::loadOrigin(float *image, int originHeight, int originWidth,
int outHeight, int outWidth) {
// convert array data from float to int image
cv::Mat originFloat = cv::Mat(originHeight, originWidth, CV_32F, image);
cv::Mat originInt = cv::Mat(cv::Size(originWidth, originHeight), CV_8UC1);
float2Image(originFloat, originInt, 0.0, 1.0);
// resize image if necessary
cv::Mat resize = cv::Mat(cv::Size(outWidth, outHeight), CV_8UC1);
if (originHeight == outHeight && originWidth == outWidth) {
originInt.copyTo(resize);
} else {
cv::resize(originInt, resize, cv::Size(outWidth, outHeight),
0, 0, cv::INTER_LINEAR);
}
return resize;
}
// generate flow and original image
void Seq::flow(int whichCh, int originHeight, int originWidth,
int outHeight, int outWidth, double bound) {
// following frames (start with one to support original flow computation)
for (int i = 1; i < length; i++) {
// load next image
std::string imageName;
if (printLabel) {
imageName = target + "/ch" + std::to_string(whichCh) + "_" +
std::to_string(i - 1) + "_" + std::to_string(label[i - 1]) + "_";
} else {
imageName = target + "/ch" + std::to_string(whichCh) + "_" +
std::to_string(i - 1) + "_";
}
// write the original image and save to file
cv::Mat image = loadOrigin(ch[whichCh] + i * originHeight * originWidth,
originHeight, originWidth, outHeight, outWidth); // resize image for out
cv::imwrite(imageName + "image.jpg", image);
}
}
================================================
FILE: image/seq.hpp
================================================
#ifndef SEQ_H
#define SEQ_H
#include
#include
// opencv
#include
#include "h5Data.hpp"
#endif
class Seq {
private:
std::string source, target;
int numChannel;
int length;
bool printLabel;
// use file pointer to avoid call destructor of h5Data too soon
h5Data *h5file = NULL;
// not in charge of manage memory
int *label;
float *ch[8];
void resize();
// void flow();
public:
Seq(std::string s, std::string t, int n, bool l);
~Seq();
inline float cast(float v, float l, float u) {
if (v > u) return 255;
else if (v < l) return 0;
else return cvRound(255 * (v - l) / (u - l));
}
void float2Image(const cv::Mat &floatMat, cv::Mat &imageMat,
double lBound, double uBound);
cv::Mat loadOrigin(float *image, int originHeight, int originWidth,
int outHeight, int outWidth);
void flow(int whichCh, int originHeight, int originWidth,
int outHeight, int outWidth, double bound);
};
================================================
FILE: net/data.lua
================================================
require 'class'
require 'paths'
require 'mattorch'
local json = require 'lunajson'
require 'net.util'
local Data = torch.class('Data')
function Data:__init(fileDir, listFile, meanFile, listType,
inputSize, inputCh, labelSize, dataSize, dataCh, ch, useCuda)
self.seqList = self:getSeq(fileDir, listFile, listType)
self.mean = self:loadMean(meanFile)
self.inputSize = inputSize
self.inputCh = inputCh
self.labelSize = labelSize
self.dataSize = dataSize
self.dataCh = dataCh
self.ch = ch
self.useCuda = useCuda
end
function Data:getSeq(fileDir, listFile, listType)
-- read file into json
local f = io.open(listFile, 'rb')
local jsonString = f:read('*all')
local jsonObject = json.decode(jsonString)
f:close()
-- arrange it according to different label
local seqList = {}
for i in pairs(jsonObject[listType]) do
seqList[#seqList + 1] = paths.concat(fileDir, jsonObject[listType][i])
end
return seqList
end
-- random shuffle a list
function Data:shuffleList(list)
local n, random = #list, math.random
for i = 1, n do
local j, k = random(n), random(n)
list[j], list[k] = list[k], list[j]
end
return list
end
-- load mat mean file
function Data:loadMean(meanFile)
return mattorch.load(meanFile)
end
function Data:initBatch(batchSize, seqLength)
-- initialize inputs and targets, zero mask will ignore null target
local inputs, targets = {}, {}
for l = 1, seqLength do
-- init with 4d data (cnn and uni)
inputs[l] = torch.Tensor(batchSize,
self.inputCh, self.inputSize, self.inputSize):zero()
targets[l] = torch.Tensor(batchSize):fill(self.labelSize)
end
return inputs, targets
end
================================================
FILE: net/imageseq.lua
================================================
require 'class'
require 'paths'
require 'image'
local json = require 'lunajson'
require 'net.seq'
require 'net.util'
local ImageSeq = torch.class('ImageSeq', 'Seq')
function ImageSeq:__init(seqName, mean, dataSize, dataCh, ch, useCuda)
Seq.__init(self, seqName, mean, dataSize, dataCh, ch, useCuda)
self.info = self:getSeqInfo()
-- load frame depends on image type, lib image support only float tensor
-- reset tensor type back to float tensor when using cuda
torch.setdefaulttensortype('torch.FloatTensor')
self.imageFrame = self:loadFrame()
-- set tensor type back to cuda tensor when using cuda
if self.useCuda then
torch.setdefaulttensortype('torch.CudaTensor')
end
end
-- read label file
function ImageSeq:getSeqInfo()
-- read file into json
local f = io.open(paths.concat(self.name, 'label.json'), 'rb')
local jsonString = f:read('*all')
local jsonObject = json.decode(jsonString)
f:close()
-- convert into num based index, 0-based to 1-based
-- label is also 1-based: 0-4 => 1-5
local seqInfo = {}
for i in pairs(jsonObject) do
seqInfo[i + 1] = jsonObject[i] + 1
end
return seqInfo
end
function ImageSeq:loadFrame()
local frame = {}
local m = self.mean[string.format('ch%d_image', self.ch)]
:permute(3, 2, 1) -- the matlab order is reversed
-- access each frame, 1-based to 0-based image name
for f = 1, #self.info do
local name = string.format('ch%d_%d_image.jpg', self.ch, f - 1)
-- for image tensor, first map from 0-1 to 0-255,
-- then minus mean (also in 0-255)
frame[#frame + 1] = (image.load(paths.concat(self.name, name), 1) * 256) - m
end
return frame
end
function ImageSeq:getImage(f)
-- appending later frames
local mat = self.imageFrame[f]
for i = 1, self.dataCh - 1 do
mat = torch.cat(mat, self.imageFrame[math.min(#self.info, f + i)], 1)
end
return mat
end
function ImageSeq:getFrame(f)
-- load data depends on image or flow
local matrix = self:getImage(f)
return matrix
end
================================================
FILE: net/main.lua
================================================
require 'net.util'
-- parse cmd parameters
local cmd = torch.CmdLine()
cmd:option('--file', '../collect_image/origin_trans', 'Image dir')
cmd:option('--list', 'tmp/file.json', 'List file json')
cmd:option('--load', '', 'Load model file')
cmd:option('--inputsize', 32, 'Input size')
cmd:option('--inputch', 4, 'Input channel')
cmd:option('--label', 3, 'Label size')
cmd:option('--datasize', 32, 'Data size')
cmd:option('--datach', 4, 'Data channel (number of stack)')
cmd:option('--batch', 4, 'Batch size')
cmd:option('--maxseq', 40, 'Sequence length')
cmd:option('--cuda', false, 'Use CUDA')
cmd:option('--cudnn', false, 'Use CUDNN')
local opt = cmd:parse(arg or {})
--- other parameters
local meanFile = 'tmp/mean_32.mat'
local ch = 1
-- default tensor
if opt.cuda then
require 'cutorch'
require 'cunn'
torch.setdefaulttensortype('torch.CudaTensor')
else
torch.setdefaulttensortype('torch.FloatTensor')
end
-- train net, switch train type
local trainObject
require 'net.rnntrain'
trainObject = RnnTrain(opt.file, opt.list, meanFile,
opt.inputsize, opt.inputch, opt.label,
opt.datasize, opt.datach, ch,
opt.load, opt.cuda, opt.cudnn)
trainObject:train(opt.batch, opt.maxseq)
================================================
FILE: net/net.lua
================================================
require 'class'
local Net = torch.class('Net')
function Net:__init(useCuda, useCudnn)
-- cuda support
self.useCuda = useCuda
self.useCudnn = useCudnn
if self.useCuda then
require 'cutorch'
require 'cunn'
if self.useCudnn then
require 'cudnn'
end
end
end
-- load already exist net
function Net:loadNet(loadFile)
print('[net] loading model ' .. loadFile)
net = torch.load(loadFile)
print(net)
return net
end
-- cuda and cudnn support
function Net:cudaNet(net)
if self.useCuda then
print('[net] with cuda')
net = net:cuda()
if self.useCudnn then
print('[net] with cudnn')
cudnn.convert(net, cudnn)
end
else
print('[net] without cuda')
end
print(net)
return net
end
================================================
FILE: net/rnndata.lua
================================================
require 'class'
require 'net.data'
require 'net.imageseq'
local RnnData = torch.class('RnnData', 'Data')
function RnnData:__init(fileDir, listFile, meanFile, listType,
inputSize, inputCh, labelSize, dataSize, dataCh, ch, useCuda)
Data.__init(self, fileDir, listFile, meanFile, listType,
inputSize, inputCh, labelSize, dataSize, dataCh, ch, useCuda)
end
function RnnData:frameSeq(seq, maxSeq)
-- load label for each frame, label is 1-based, f is frame id
-- offset introduce zero paddings at the beginning of the sequence
local frames = {}
if #seq.info < maxSeq then
local offset = maxSeq - #seq.info
for f = 1, #seq.info do
frames[offset + f] = f
end
else
for f = 1, maxSeq do
frames[f] = math.floor(f * (#seq.info / maxSeq))
end
end
return frames
end
-- load batch to tensor
function RnnData:loadBatch(batchNum, batchSize, maxSeq)
local inputs, targets = self:initBatch(batchSize, maxSeq)
-- load data into inputs and targets
local batchStart = (batchNum - 1) * batchSize
for b = 1, batchSize do
-- get seq info
local seq
seq = ImageSeq(self.seqList[batchStart + b], self.mean,
self.dataSize, self.dataCh, self.ch, self.useCuda)
-- generate frame sequence
local frames = self:frameSeq(seq, maxSeq)
for i, v in pairs(frames) do
inputs[i][b] = seq:getFrame(v)
targets[i][b] = seq.info[v]
end
end
return inputs, targets
end
================================================
FILE: net/rnntrain.lua
================================================
require 'class'
require 'net.train'
require 'net.util'
local RnnTrain = torch.class('RnnTrain', 'Train')
function RnnTrain:__init(fileDir, listFile, meanFile,
inputSize, inputCh, labelSize, dataSize, dataCh, ch,
loadFile, useCuda, useCudnn)
-- call base train constructor
Train.__init(self, labelSize, useCuda, useCudnn)
-- load data
require 'net.rnndata'
self.evalData = RnnData(fileDir, listFile, meanFile, 'eval',
inputSize, inputCh, labelSize, dataSize, dataCh, ch, useCuda)
print(string.format('[eval] data with %d seq', #self.evalData.seqList))
-- build net
local netObject
require 'net.uninet'
netObject = UniNet(useCuda, useCudnn)
self.net = netObject:loadNet(loadFile)
end
function RnnTrain:batchEval(b, inputs)
-- evaluate forward as whole sequence
self.net:evaluate()
local outputs = self.net:forward(self:convertCuda(inputs))
self.net:forget()
return outputs
end
================================================
FILE: net/seq.lua
================================================
require 'class'
require 'paths'
require 'image'
local json = require 'lunajson'
require 'net.util'
local Seq = torch.class('Seq')
function Seq:__init(seqName, mean, dataSize, dataCh, ch, useCuda)
-- load info
self.name = seqName
self.mean = mean
self.dataSize = dataSize
self.dataCh = dataCh
self.ch = ch
self.useCuda = useCuda
end
================================================
FILE: net/stat.lua
================================================
require 'class'
local json = require 'lunajson'
local Stat = torch.class('Stat')
function Stat:__init(labelSize, batchSize, maxSeq)
self.labelSize = labelSize
self.batchSize = batchSize
self.maxSeq = maxSeq
-- init confusion matrix (correct label (size) * predict label (size))
self.confus = self:initMat(labelSize, labelSize)
end
function Stat:initMat(height, width)
local mat = {}
for h = 1, height do
mat[h] = {}
for w = 1, width do
mat[h][w] = 0
end
end
return mat
end
function Stat:updateStat(outputs, targets)
-- calculate one sequence by one sequence
for o = 1, #outputs do
local _, indices = torch.max(outputs[o], 2)
for b = 1, self.batchSize do
local right, pred = targets[o][b], indices[b][1]
-- update confusion matrix
self.confus[right][pred] = self.confus[right][pred] + 1
end
end
end
function Stat:sum(list, length)
local result = 0
-- ignore null label in sum
for i = 1, length do
result = result + list[i]
end
return result
end
-- calculate per label stat
function Stat:calPerLabel()
local right, count = {}, {}
-- iterate for all label
for l = 1, self.labelSize do
right[l] = self.confus[l][l]
count[l] = self:sum(self.confus[l], #self.confus[l])
end
return right, count
end
-- print matrix
function Stat:printMat(str, right, count, matRight, matCount)
for l = 1, #right do
if count[l] > 0 then
local accLine = right[l] * 100 / count[l]
io.write(string.format(' %s %02d: %02.0f [', str, l, accLine))
for ll = 1, #matRight[1] do
-- print each element depends on the type and value
if type(matCount[l]) == 'number' and matCount[l] > 0 then
local acc = matRight[l][ll] * 100 / matCount[l]
io.write(string.format(' %02.0f ', acc))
else
io.write(' -- ')
end
end
io.write(']\n')
else
io.write(string.format(' %s %02d: --\n', str, l))
end
end
end
function Stat:print(type)
local right, count = self:calPerLabel()
-- calculate sum of label
local accEpoch = self:sum(right, #right - 1) / self:sum(count, #count - 1)
io.write(string.format('[%s] accuracy %f\n', type, accEpoch))
-- print per label and confusion matrix
self:printMat('label', right, count, self.confus, count)
-- return accuracy for lr update
return accEpoch
end
================================================
FILE: net/train.lua
================================================
require 'class'
local Train = torch.class('Train')
function Train:__init(labelSize, useCuda, useCudnn)
-- require and cuda support
require 'net.stat'
self.useCuda = useCuda
self.useCudnn = useCudnn
if useCuda then
require 'cutorch'
require 'cunn'
end
-- other parameters
self.labelSize = labelSize
end
-- one step of evaluate to all data
function Train:epochEval(batchSize, maxSeq)
local stat = Stat(self.labelSize, batchSize, maxSeq)
for b = 1, math.floor(#self.evalData.seqList / batchSize) do
-- load data, start evaluating, depends on net type
local inputs, targets = self.evalData:loadBatch(b, batchSize, maxSeq)
local outputs = self:batchEval(b, inputs)
-- update confusion matrix
stat:updateStat(outputs, targets)
end
return stat:print('eval')
end
-- convert to cuda
function Train:convertCuda(list)
if self.cuda then
-- convert table one by one to cuda
for l = 1, #list do
list[l] = list[l]:cuda()
end
return list
else
return list
end
end
function Train:train(batchSize, maxSeq)
-- evaluate by epoch
self:epochEval(batchSize, maxSeq)
print('Finished')
end
================================================
FILE: net/uninet.lua
================================================
require 'class'
require 'rnn'
require 'net.net'
local UniNet = torch.class('UniNet', 'Net')
function UniNet:__init(useCuda, useCudnn)
Net.__init(self, useCuda, useCudnn)
end
================================================
FILE: net/util.lua
================================================
-- helper function to check if string start with some sub-string
function string:startsWith(start)
return string.sub(self, 1, string.len(start)) == start
end
function string:split(sep)
local sep, fields = sep or ":", {}
local pattern = string.format("([^%s]+)", sep)
self:gsub(pattern, function(c) fields[#fields+1] = c end)
return fields
end
================================================
FILE: pre/image.py
================================================
import cv2
import numpy as np
class image:
def __init__(self, file_name):
self.file_name = file_name
# load image and resize image with size
def load(self, height, width):
frame = cv2.imread(self.file_name, cv2.IMREAD_GRAYSCALE)
if height and width:
frame = cv2.resize(frame, (width, height))
frame = np.expand_dims(frame, axis=0).astype('float32')
return frame
================================================
FILE: pre/main.py
================================================
import argparse
from seq import Seq
# call different method depends on op
def main(op, file_dir, target_dir, num_channel=1, redo=False,
origin_size=32, out_size=32):
s = Seq()
if op == 'image':
s.generate_image(file_dir, target_dir, num_channel,
origin_size, out_size, redo)
elif op == 'clean':
s.clean_image(file_dir)
elif op == 'mean':
s.generate_mean(file_dir, target_dir, num_channel, out_size)
else:
raise NameError('The operation type {} is undefined'.format(op))
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Generate image")
parser.add_argument('--op', type=str)
parser.add_argument('--file', type=str)
parser.add_argument('--target', type=str)
parser.add_argument('--channel', type=int)
parser.add_argument('--redo', action='store_true')
parser.add_argument('--originsize', type=int)
parser.add_argument('--outsize', type=int)
args = parser.parse_args()
main(args.op, args.file, args.target, args.channel, args.redo,
args.originsize, args.outsize)
================================================
FILE: pre/op.py
================================================
import os, subprocess
import numpy as np
import h5py
import json
from image import image
class Operation:
def __init__(self):
pass
# generate image for each sequence
def generate_image(self, source, target, num_channel, origin_size=32,
out_size=32, bound=15.0, bin_image='image/bin/image'):
if not os.path.exists(target): os.makedirs(target)
# run flow command
p = subprocess.Popen([bin_image,
'--source', source, '--target', target,
'--channel', str(num_channel),
'--origin', str(origin_size),
'--out', str(out_size),
'--bound', str(bound)])
p.wait()
# generate label file
self._generate_label(source, target)
# load labels
def _generate_label(self, source, target, label_file='label.json'):
with h5py.File(source, 'r') as hf:
labels = [int(l) for l in hf['label'][()]]
# save labels as a dict in json file
labels_dict = {i: l
for i, l in zip(range(len(labels) - 1), labels[1:])}
with open(os.path.join(target, label_file), 'w') as jf:
json.dump(labels_dict, jf)
# delete generated image files
def clean_image(self, file_dir, label_file='label.json'):
print 'Cleaning', file_dir
# delete images in dir
image_files = [os.path.join(file_dir, f)
for f in os.listdir(file_dir)
if os.path.isfile(os.path.join(file_dir, f)) and 'jpg' in f]
for image_file in image_files:
try:
os.remove(image_file)
except Exception, e:
print e
# delete label file
try:
os.remove(os.path.join(file_dir, label_file))
except Exception, e:
print e
# delete dir
try:
os.rmdir(file_dir)
except Exception, e:
print e
# get frame count
def _get_frame_num(self, source, label_file='label.json'):
with open(os.path.join(source, label_file), 'r') as jf:
labels_dict = json.load(jf)
return len(labels_dict)
# setup mean counter and accumulator at the beginning
def setup_mean(self, num_channel, out_size):
self.image_sums = {}
self.count = 0.0
for c in range(num_channel):
self.image_sums['ch%i_image' % (c,)] = \
np.zeros((1, out_size, out_size), dtype='float32')
# accumulate mean for each sequence
def accum_mean(self, source, num_channel, out_size):
print 'Loading mean', source
frame_num = self._get_frame_num(source)
self.count += frame_num
for c in range(num_channel):
for i in range(frame_num):
image_name = os.path.join(source,
'ch%i_%i_image.jpg' % (c, i))
self.image_sums['ch%i_image' % (c,)] += \
image(image_name).load(out_size, out_size)
# save accumulated mean to file
def save_mean(self, mean_file, num_channel):
# store file as hdf5
if mean_file.endswith('h5'):
print 'Save as hdf5'
with h5py.File(mean_file, 'w') as f:
for c in range(num_channel):
f.create_dataset('ch%i_image' % (c,),
data=self.image_sums['ch%i_image' % (c,)]
/ self.count)
# store file as matlab data
elif mean_file.endswith('mat'):
import scipy.io as sio
print 'Save as mat'
data = {}
for c in range(num_channel):
data['ch%i_image' % (c,)] = \
self.image_sums['ch%i_image' % (c,)] / self.count
sio.savemat(mean_file, data)
================================================
FILE: pre/seq.py
================================================
import os, re
import numpy as np
import h5py
from op import Operation
class Seq:
def __init__(self):
pass
# get files in label directories
def get_h5(self, file_dir):
return [os.path.join(file_dir, f)
for f in os.listdir(file_dir)
if os.path.isfile(os.path.join(file_dir, f)) and 'h5' in f]
# get list of target image dir names
def get_dir(self, file_dir):
return [os.path.join(file_dir, d)
for d in os.listdir(file_dir)
if os.path.isdir(os.path.join(file_dir, d))]
# generate image
def generate_image(self, file_dir, target_dir, num_channel,
origin_size, out_size, redo):
source_files = self.get_h5(file_dir)
o = Operation()
for s in source_files:
t = os.path.join(target_dir, re.sub('.h5', '',
os.path.basename(s)))
if redo or not os.path.exists(t):
print 'Generating', s
o.generate_image(s, t, num_channel, origin_size, out_size)
# clean image
def clean_image(self, file_dir):
file_dirs = self.get_dir(file_dir)
o = Operation()
for d in file_dirs:
o.clean_image(d)
# generate mean
def generate_mean(self, file_dir, target, num_channel, out_size):
o = Operation()
o.setup_mean(num_channel, out_size)
source_dir = self.get_dir(file_dir)
for s in source_dir:
o.accum_mean(s, num_channel, out_size)
o.save_mean(target, num_channel)