Showing preview only (254K chars total). Download the full file or copy to clipboard to get everything.
Repository: simonalexanderson/ListenDenoiseAction
Branch: main
Commit: 9b0885b69991
Files: 65
Total size: 238.9 KB
Directory structure:
gitextract_9cw0czva/
├── .gitignore
├── Dockerfile
├── LICENSE
├── README.md
├── data/
│ └── motorica_dance/
│ ├── audio18_features.txt
│ ├── audio29_features.txt
│ ├── ch0_spec_beatact_features.txt
│ ├── dance_styles.txt
│ ├── dance_test_files.txt
│ ├── dance_train_files.txt
│ ├── data_pipe.expmap_30fps.sav
│ ├── gen_files.txt
│ ├── kthstreet_gKR_sFM_cAll_d01_mKR_ch01_chargedcableupyour_001_00.audio29_30fps.pkl
│ ├── kthstreet_gKR_sFM_cAll_d01_mKR_ch01_chargedcableupyour_001_00.expmap_30fps.pkl
│ ├── kthstreet_gKR_sFM_cAll_d01_mKR_ch01_chargedcableupyour_001_01.audio29_30fps.pkl
│ ├── kthstreet_gKR_sFM_cAll_d01_mKR_ch01_chargedcableupyour_001_01.expmap_30fps.pkl
│ ├── kthstreet_gLH_sFM_cAll_d02_mLH_ch01_lala_001_00.audio29_30fps.pkl
│ ├── kthstreet_gLH_sFM_cAll_d02_mLH_ch01_lala_001_00.expmap_30fps.pkl
│ ├── kthstreet_gLH_sFM_cAll_d02_mLH_ch01_lala_001_01.audio29_30fps.pkl
│ ├── kthstreet_gLH_sFM_cAll_d02_mLH_ch01_lala_001_01.expmap_30fps.pkl
│ ├── kthstreet_gLO_sFM_cAll_d02_mLO_ch01_arethafranklinrocksteady_002_00.audio29_30fps.pkl
│ ├── kthstreet_gLO_sFM_cAll_d02_mLO_ch01_arethafranklinrocksteady_002_00.expmap_30fps.pkl
│ ├── kthstreet_gLO_sFM_cAll_d02_mLO_ch01_arethafranklinrocksteady_002_01.audio29_30fps.pkl
│ ├── kthstreet_gLO_sFM_cAll_d02_mLO_ch01_arethafranklinrocksteady_002_01.expmap_30fps.pkl
│ ├── kthstreet_gPO_sFM_cAll_d01_mPO_ch01_bombom_002_00.audio29_30fps.pkl
│ ├── kthstreet_gPO_sFM_cAll_d01_mPO_ch01_bombom_002_00.expmap_30fps.pkl
│ ├── kthstreet_gPO_sFM_cAll_d01_mPO_ch01_bombom_002_01.audio29_30fps.pkl
│ ├── kthstreet_gPO_sFM_cAll_d01_mPO_ch01_bombom_002_01.expmap_30fps.pkl
│ ├── kthstreet_gPO_sFM_cAll_d02_mPO_ch01_bombom_001_00.audio29_30fps.pkl
│ ├── kthstreet_gPO_sFM_cAll_d02_mPO_ch01_bombom_001_00.expmap_30fps.pkl
│ ├── kthstreet_gPO_sFM_cAll_d02_mPO_ch01_bombom_001_01.audio29_30fps.pkl
│ ├── kthstreet_gPO_sFM_cAll_d02_mPO_ch01_bombom_001_01.expmap_30fps.pkl
│ └── pose_features.expmap.txt
├── experiments/
│ ├── dance_LDA-U.sh
│ ├── dance_LDA.sh
│ └── dance_mix_experts.sh
├── hparams/
│ ├── diffusion_dance_LDA-U.yaml
│ └── diffusion_dance_LDA.yaml
├── models/
│ ├── BaseModel.py
│ ├── LightningModel.py
│ ├── nn.py
│ └── transformer/
│ ├── tisa_transformer.py
│ └── tisa_v2.py
├── pretrained_models/
│ ├── .gitkeep
│ └── README.md
├── pymo/
│ ├── Pivots.py
│ ├── Quaternions.py
│ ├── __init__.py
│ ├── data.py
│ ├── features.py
│ ├── parsers.py
│ ├── preprocessing.py
│ ├── rotation_tools.py
│ ├── viz_tools.py
│ └── writers.py
├── requirements.txt
├── run_docker.sh
├── synthesize.py
├── train.py
└── utils/
├── cut_wav.py
├── download_from_youtube.py
├── duplicate_features.sh
├── hparams.py
├── logging_mixin.py
└── motion_dataset.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
results/*
**/__pycache__
================================================
FILE: Dockerfile
================================================
FROM nvcr.io/nvidia/pytorch:22.10-py3
ENV DEBIAN_FRONTEND=noninteractive
WORKDIR /workspace
RUN apt-get update
RUN apt-get install rename
RUN apt-get install -y ffmpeg
COPY requirements.txt /tmp
RUN pip install -r /tmp/requirements.txt
================================================
FILE: LICENSE
================================================
License and Copyright Information
---------------------------------
The contents of this repository may not be used for any purpose other than academic research. It is free to use for research purposes by academic institutes, companies, and individuals. Use for commercial purposes is not permitted without prior written consent from Motorica AB. If you are interested in using the codebase, pretrained models or the dataset for commercial purposes or non-research purposes, please contact us at info@motorica.ai in advance. Unauthorised redistribution is prohibited without written approval.
Please include the following citations in any preprints and publications that use this repository.
@article{alexanderson2023listen,
title={Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models},
author={Alexanderson, Simon and Nagy, Rajmund and Beskow, Jonas and Henter, Gustav Eje},
journal={ACM Trans. Graph.},
volume={42},
number={4},
pages={1--20},
doi={10.1145/3592458},
year={2023}
}
The code for translation-invariant self-attention (TISA) was written by Ulme Wennberg. Please cite their ACL 2021 article if you use this code.
================================================
FILE: README.md
================================================
# Listen, Denoise, Action!
This repository provides code and models for the paper [Listen, denoise, action! Audio-driven motion synthesis with diffusion models](https://arxiv.org/abs/2211.09707).
Please watch the following video for an introduction to our work:
* [SIGGRAPH 2023 presentation](https://youtu.be/Qfd2EpzWgok)
For video samples and a general overview, please see [our project page](https://www.speech.kth.se/research/listen-denoise-action/). For the new dance dataset with high-quality mocap, please see [the Motorica Dance Dataset](https://github.com/simonalexanderson/MotoricaDanceDataset/).
## Installation
We provide a Docker file and `requirements.txt` for installation using a Docker image or Conda.
### Installation using Conda
```
conda install python=3.9
conda install -c conda-forge mpi4py mpich
pip install -r requirements.txt
```
## Dance synthesis demo
### Data and pretrained models
Please [download our pretrained dance models here](https://zenodo.org/record/8156769) and move them to the `pretrained_models` folder.
We include processed music inputs from the test dataset in the `data` folder for generating dances from the model.
### Synthesis scripts
You can use the following shell scripts for reproducing the dance user studies in the paper:
```
./experiments/dance_LDA.sh
./experiments/dance_LDA-U.sh
```
To try out locomotion synthesis, please go to https://www.motorica.ai/.
## Training data
The four main training datasets from our SIGGRAPH 2023 paper are available online:
* [The Trinity Speech Gesture Dataset](https://trinityspeechgesture.scss.tcd.ie/)
* [The ZEGGS dataset](https://github.com/ubisoft/ubisoft-laforge-ZeroEGGS)
* [The 100STYLE dataset](https://www.ianxmason.com/100style/)
* [The Motorica Dance Dataset](https://github.com/simonalexanderson/MotoricaDanceDataset/), a new dataset with high-quality dance mocap released together with our paper
## License and copyright information
The contents of this repository may not be used for any purpose other than academic research. It is free to use for research purposes by academic institutes, companies, and individuals. Use for commercial purposes is not permitted without prior written consent from Motorica AB. If you are interested in using the codebase, pretrained models or the dataset for commercial purposes or non-research purposes, please contact us at info@motorica.ai in advance. Unauthorised redistribution is prohibited without written approval.
### Attribution
Please include the following citations in any preprints and publications that use this repository.
```
@article{alexanderson2023listen,
title={Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models},
author={Alexanderson, Simon and Nagy, Rajmund and Beskow, Jonas and Henter, Gustav Eje},
year={2023}
issue_date={August 2023},
publisher={ACM},
volume={42},
number={4},
doi={10.1145/3592458},
journal={ACM Trans. Graph.},
articleno={44},
numpages={20},
pages={44:1--44:20}
}
```
The [code for translation-invariant self-attention](https://github.com/ulmewennberg/tisa) (TISA) was written by [Ulme Wennberg](https://www.kth.se/profile/ulme). Please cite [the correspoding ACL 2021 article](https://aclanthology.org/2021.acl-short.18) if you use this code.
================================================
FILE: data/motorica_dance/audio18_features.txt
================================================
MFCC_0
MFCC_1
MFCC_2
Chroma_0
Chroma_1
Chroma_2
Chroma_3
Chroma_4
Chroma_5
Chroma_6
Chroma_7
Chroma_8
Chroma_9
Chroma_10
Chroma_11
Spectralflux_0
Beatactivation_0
Beat_0
================================================
FILE: data/motorica_dance/audio29_features.txt
================================================
MFCC_0
MFCC_1
MFCC_2
MFCC_3
MFCC_4
MFCC_5
MFCC_6
MFCC_7
MFCC_8
MFCC_9
MFCC_10
MFCC_11
MFCC_12
MFCC_13
MFCC_14
MFCC_15
MFCC_16
MFCC_17
MFCC_18
MFCC_19
Chroma_0
Chroma_1
Chroma_2
Chroma_3
Chroma_4
Chroma_5
Spectralflux_0
Beatactivation_0
Beat_0
================================================
FILE: data/motorica_dance/ch0_spec_beatact_features.txt
================================================
Chroma_0
Spectralflux_0
Beatactivation_0
================================================
FILE: data/motorica_dance/dance_styles.txt
================================================
gCA
gCH
gJZ
gKR
gLH
gLO
gPO
gTP
================================================
FILE: data/motorica_dance/dance_test_files.txt
================================================
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_whitemanpaulandhisorchestraloisiana_006_00
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_whitemanpaulandhisorchestraloisiana_006_01
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_bennygoodmansugarfootstomp_003_00
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_bennygoodmansugarfootstomp_003_01
kthjazz_gTP_sFM_sngl_d02_015_00
kthjazz_gTP_sFM_sngl_d02_015_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch24_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch24_01
kthstreet_gKR_sFM_cAll_d01_mKR_ch01_chargedcableupyour_001_00
kthstreet_gKR_sFM_cAll_d01_mKR_ch01_chargedcableupyour_001_01
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_thisisit_001_00
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_thisisit_001_01
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_lala_001_00
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_lala_001_01
kthstreet_gLO_sFM_cAll_d02_mLO_ch01_arethafranklinrocksteady_002_00
kthstreet_gLO_sFM_cAll_d02_mLO_ch01_arethafranklinrocksteady_002_01
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_bombom_002_00
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_bombom_002_01
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_bombom_001_00
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_bombom_001_01
================================================
FILE: data/motorica_dance/dance_train_files.txt
================================================
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_beatlestreetwashboardbandfortyandtight_003_00
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_beatlestreetwashboardbandfortyandtight_003_00_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_beatlestreetwashboardbandfortyandtight_003_01
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_beatlestreetwashboardbandfortyandtight_003_01_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_charlestonchaserswabashblues_004_00
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_charlestonchaserswabashblues_004_00_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_charlestonchaserswabashblues_004_01
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_charlestonchaserswabashblues_004_01_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_lloydkeatingandhismusicturnontheheat_010_00
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_lloydkeatingandhismusicturnontheheat_010_00_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_lloydkeatingandhismusicturnontheheat_010_01
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_lloydkeatingandhismusicturnontheheat_010_01_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_royalgardenblues_002_00
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_royalgardenblues_002_00_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_royalgardenblues_002_01
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_royalgardenblues_002_01_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_tedlewisgladragdoll_007_00
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_tedlewisgladragdoll_007_00_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_tedlewisgladragdoll_007_01
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_tedlewisgladragdoll_007_01_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_therythmiceightkansascitykitty_001_00
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_therythmiceightkansascitykitty_001_00_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_therythmiceightkansascitykitty_001_01
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_therythmiceightkansascitykitty_001_01_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_therythmiceightumthaumthadadada_009_00
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_therythmiceightumthaumthadadada_009_00_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_therythmiceightumthaumthadadada_009_01
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_therythmiceightumthaumthadadada_009_01_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_thesavoyorpheansfivefoottwoeyesofblue_008_00
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_thesavoyorpheansfivefoottwoeyesofblue_008_00_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_thesavoyorpheansfivefoottwoeyesofblue_008_01
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_thesavoyorpheansfivefoottwoeyesofblue_008_01_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_thesavoyorpheansthecharleston_005_00
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_thesavoyorpheansthecharleston_005_00_mirrored
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_thesavoyorpheansthecharleston_005_01
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_thesavoyorpheansthecharleston_005_01_mirrored
kthjazz_gCH_sFM_sngl_d01_007_00
kthjazz_gCH_sFM_sngl_d01_007_00_mirrored
kthjazz_gCH_sFM_sngl_d01_007_01
kthjazz_gCH_sFM_sngl_d01_007_01_mirrored
kthjazz_gCH_sFM_sngl_d01_008_00
kthjazz_gCH_sFM_sngl_d01_008_00_mirrored
kthjazz_gCH_sFM_sngl_d01_008_01
kthjazz_gCH_sFM_sngl_d01_008_01_mirrored
kthjazz_gCH_sFM_sngl_d01_009_00
kthjazz_gCH_sFM_sngl_d01_009_00_mirrored
kthjazz_gCH_sFM_sngl_d01_009_01
kthjazz_gCH_sFM_sngl_d01_009_01_mirrored
kthjazz_gCH_sFM_sngl_d01_019_00
kthjazz_gCH_sFM_sngl_d01_019_00_mirrored
kthjazz_gCH_sFM_sngl_d01_019_01
kthjazz_gCH_sFM_sngl_d01_019_01_mirrored
kthjazz_gCH_sFM_sngl_d02_005_00
kthjazz_gCH_sFM_sngl_d02_005_00_mirrored
kthjazz_gCH_sFM_sngl_d02_005_01
kthjazz_gCH_sFM_sngl_d02_005_01_mirrored
kthjazz_gCH_sFM_sngl_d02_006_00
kthjazz_gCH_sFM_sngl_d02_006_00_mirrored
kthjazz_gCH_sFM_sngl_d02_006_01
kthjazz_gCH_sFM_sngl_d02_006_01_mirrored
kthjazz_gCH_sFM_sngl_d02_018_00
kthjazz_gCH_sFM_sngl_d02_018_00_mirrored
kthjazz_gCH_sFM_sngl_d02_018_01
kthjazz_gCH_sFM_sngl_d02_018_01_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_bennygoodmanairmailspecial_002_00
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_bennygoodmanairmailspecial_002_00_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_bennygoodmanairmailspecial_002_01
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_bennygoodmanairmailspecial_002_01_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_budpowellboundingwithbud_006_00
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_budpowellboundingwithbud_006_00_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_budpowellboundingwithbud_006_01
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_budpowellboundingwithbud_006_01_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_countbasieshortygeorge_007_00
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_countbasieshortygeorge_007_00_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_countbasieshortygeorge_007_01
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_countbasieshortygeorge_007_01_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_dizzygillespiegroovinghigh_004_00
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_dizzygillespiegroovinghigh_004_00_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_dizzygillespiegroovinghigh_004_01
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_dizzygillespiegroovinghigh_004_01_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_dizzygillespieohbopshbam_005_00
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_dizzygillespieohbopshbam_005_00_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_dizzygillespieohbopshbam_005_01
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_dizzygillespieohbopshbam_005_01_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_dukeellingtontaketheatrain_001_00
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_dukeellingtontaketheatrain_001_00_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_dukeellingtontaketheatrain_001_01
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_dukeellingtontaketheatrain_001_01_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_glennmillerinthemood_003_00
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_glennmillerinthemood_003_00_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_glennmillerinthemood_003_01
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_glennmillerinthemood_003_01_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_jimmyluncefordfordancersonly_004_00
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_jimmyluncefordfordancersonly_004_00_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_jimmyluncefordfordancersonly_004_01
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_jimmyluncefordfordancersonly_004_01_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_lionelhamptonflyinghome_005_00
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_lionelhamptonflyinghome_005_00_mirrored
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_lionelhamptonflyinghome_005_01
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_lionelhamptonflyinghome_005_01_mirrored
kthjazz_gJZ_sFM_sngl_d01_001_00
kthjazz_gJZ_sFM_sngl_d01_001_00_mirrored
kthjazz_gJZ_sFM_sngl_d01_001_01
kthjazz_gJZ_sFM_sngl_d01_001_01_mirrored
kthjazz_gJZ_sFM_sngl_d01_002_00
kthjazz_gJZ_sFM_sngl_d01_002_00_mirrored
kthjazz_gJZ_sFM_sngl_d01_002_01
kthjazz_gJZ_sFM_sngl_d01_002_01_mirrored
kthjazz_gJZ_sFM_sngl_d01_020_00
kthjazz_gJZ_sFM_sngl_d01_020_00_mirrored
kthjazz_gJZ_sFM_sngl_d01_020_01
kthjazz_gJZ_sFM_sngl_d01_020_01_mirrored
kthjazz_gJZ_sFM_sngl_d01_021_00
kthjazz_gJZ_sFM_sngl_d01_021_00_mirrored
kthjazz_gJZ_sFM_sngl_d01_021_01
kthjazz_gJZ_sFM_sngl_d01_021_01_mirrored
kthjazz_gJZ_sFM_sngl_d02_003_00
kthjazz_gJZ_sFM_sngl_d02_003_00_mirrored
kthjazz_gJZ_sFM_sngl_d02_003_01
kthjazz_gJZ_sFM_sngl_d02_003_01_mirrored
kthjazz_gJZ_sFM_sngl_d02_004_00
kthjazz_gJZ_sFM_sngl_d02_004_00_mirrored
kthjazz_gJZ_sFM_sngl_d02_004_01
kthjazz_gJZ_sFM_sngl_d02_004_01_mirrored
kthjazz_gTP_sFM_sngl_d01_010_00
kthjazz_gTP_sFM_sngl_d01_010_00_mirrored
kthjazz_gTP_sFM_sngl_d01_010_01
kthjazz_gTP_sFM_sngl_d01_010_01_mirrored
kthjazz_gTP_sFM_sngl_d02_014_00
kthjazz_gTP_sFM_sngl_d02_014_00_mirrored
kthjazz_gTP_sFM_sngl_d02_014_01
kthjazz_gTP_sFM_sngl_d02_014_01_mirrored
kthjazz_gTP_sFM_sngl_d02_016_00
kthjazz_gTP_sFM_sngl_d02_016_00_mirrored
kthjazz_gTP_sFM_sngl_d02_016_01
kthjazz_gTP_sFM_sngl_d02_016_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch0_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch0_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch0_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch0_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch10_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch10_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch10_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch10_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch13_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch13_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch13_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch13_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch14_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch14_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch14_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch14_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch15_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch15_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch15_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch15_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch16_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch16_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch16_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch16_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch17_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch17_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch17_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch17_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch19_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch19_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch19_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch19_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch1_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch1_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch1_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch1_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch20_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch20_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch20_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch20_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch22_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch22_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch22_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch22_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch24_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch24_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch24_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch24_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch28_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch28_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch28_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch28_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch29_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch29_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch29_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch29_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch2_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch2_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch2_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch2_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch30_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch30_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch30_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch30_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch3_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch3_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch3_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch3_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch4_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch4_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch4_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch4_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch5_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch5_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch5_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch5_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch6_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch6_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch6_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch6_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch8_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch8_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch8_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch8_01_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch9_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch9_00_mirrored
kthmisc_gCA_sFM_cAll_d01_mCA_ch9_01
kthmisc_gCA_sFM_cAll_d01_mCA_ch9_01_mirrored
kthstreet_gKR_sFM_cAll_d01_mKR1_ch01_00
kthstreet_gKR_sFM_cAll_d01_mKR1_ch01_00_mirrored
kthstreet_gKR_sFM_cAll_d01_mKR1_ch01_01
kthstreet_gKR_sFM_cAll_d01_mKR1_ch01_01_mirrored
kthstreet_gKR_sFM_cAll_d01_mKR3_ch03_00
kthstreet_gKR_sFM_cAll_d01_mKR3_ch03_00_mirrored
kthstreet_gKR_sFM_cAll_d01_mKR3_ch03_01
kthstreet_gKR_sFM_cAll_d01_mKR3_ch03_01_mirrored
kthstreet_gKR_sFM_cAll_d01_mKR4_ch04_00
kthstreet_gKR_sFM_cAll_d01_mKR4_ch04_00_mirrored
kthstreet_gKR_sFM_cAll_d01_mKR4_ch04_01
kthstreet_gKR_sFM_cAll_d01_mKR4_ch04_01_mirrored
kthstreet_gKR_sFM_cAll_d01_mKR5_ch05_00
kthstreet_gKR_sFM_cAll_d01_mKR5_ch05_00_mirrored
kthstreet_gKR_sFM_cAll_d01_mKR5_ch05_01
kthstreet_gKR_sFM_cAll_d01_mKR5_ch05_01_mirrored
kthstreet_gKR_sFM_cAll_d01_mKR6_ch06_00
kthstreet_gKR_sFM_cAll_d01_mKR6_ch06_00_mirrored
kthstreet_gKR_sFM_cAll_d01_mKR6_ch06_01
kthstreet_gKR_sFM_cAll_d01_mKR6_ch06_01_mirrored
kthstreet_gKR_sFM_cAll_d01_mKR7_ch07_00
kthstreet_gKR_sFM_cAll_d01_mKR7_ch07_00_mirrored
kthstreet_gKR_sFM_cAll_d01_mKR7_ch07_01
kthstreet_gKR_sFM_cAll_d01_mKR7_ch07_01_mirrored
kthstreet_gKR_sFM_cAll_d01_mKR_ch01_1stafterthecaos_000_00
kthstreet_gKR_sFM_cAll_d01_mKR_ch01_1stafterthecaos_000_00_mirrored
kthstreet_gKR_sFM_cAll_d01_mKR_ch01_1stafterthecaos_000_01
kthstreet_gKR_sFM_cAll_d01_mKR_ch01_1stafterthecaos_000_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH11_ch11_00
kthstreet_gLH_sFM_cAll_d01_mLH11_ch11_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH11_ch11_01
kthstreet_gLH_sFM_cAll_d01_mLH11_ch11_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH12_ch12_00
kthstreet_gLH_sFM_cAll_d01_mLH12_ch12_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH12_ch12_01
kthstreet_gLH_sFM_cAll_d01_mLH12_ch12_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH13_ch13_00
kthstreet_gLH_sFM_cAll_d01_mLH13_ch13_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH13_ch13_01
kthstreet_gLH_sFM_cAll_d01_mLH13_ch13_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH14_ch14_00
kthstreet_gLH_sFM_cAll_d01_mLH14_ch14_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH14_ch14_01
kthstreet_gLH_sFM_cAll_d01_mLH14_ch14_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH15_ch15_00
kthstreet_gLH_sFM_cAll_d01_mLH15_ch15_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH15_ch15_01
kthstreet_gLH_sFM_cAll_d01_mLH15_ch15_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH1_ch01_00
kthstreet_gLH_sFM_cAll_d01_mLH1_ch01_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH1_ch01_01
kthstreet_gLH_sFM_cAll_d01_mLH1_ch01_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH2_ch02_00
kthstreet_gLH_sFM_cAll_d01_mLH2_ch02_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH2_ch02_01
kthstreet_gLH_sFM_cAll_d01_mLH2_ch02_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH3_ch03_00
kthstreet_gLH_sFM_cAll_d01_mLH3_ch03_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH3_ch03_01
kthstreet_gLH_sFM_cAll_d01_mLH3_ch03_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH4_ch04_00
kthstreet_gLH_sFM_cAll_d01_mLH4_ch04_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH4_ch04_01
kthstreet_gLH_sFM_cAll_d01_mLH4_ch04_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH6_ch06_00
kthstreet_gLH_sFM_cAll_d01_mLH6_ch06_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH6_ch06_01
kthstreet_gLH_sFM_cAll_d01_mLH6_ch06_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH7_ch07_00
kthstreet_gLH_sFM_cAll_d01_mLH7_ch07_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH7_ch07_01
kthstreet_gLH_sFM_cAll_d01_mLH7_ch07_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH8_ch08_00
kthstreet_gLH_sFM_cAll_d01_mLH8_ch08_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH8_ch08_01
kthstreet_gLH_sFM_cAll_d01_mLH8_ch08_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH9_ch09_00
kthstreet_gLH_sFM_cAll_d01_mLH9_ch09_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH9_ch09_01
kthstreet_gLH_sFM_cAll_d01_mLH9_ch09_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_danstrams_00
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_danstrams_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_danstrams_01
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_danstrams_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_feelingsvstech_001_00
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_feelingsvstech_001_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_feelingsvstech_001_01
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_feelingsvstech_001_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_imhere20_005_00
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_imhere20_005_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_imhere20_005_01
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_imhere20_005_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_jesusbreak_001_00
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_jesusbreak_001_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_jesusbreak_001_01
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_jesusbreak_001_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_vibeness108bpm_001_00
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_vibeness108bpm_001_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_vibeness108bpm_001_01
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_vibeness108bpm_001_01_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_yougonnaregretit_003_00
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_yougonnaregretit_003_00_mirrored
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_yougonnaregretit_003_01
kthstreet_gLH_sFM_cAll_d01_mLH_ch01_yougonnaregretit_003_01_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_basketboll_002_00
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_basketboll_002_00_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_basketboll_002_01
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_basketboll_002_01_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_danstrams_000_00
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_danstrams_000_00_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_danstrams_000_01
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_danstrams_000_01_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_digit_001_00
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_digit_001_00_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_digit_001_01
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_digit_001_01_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_glitter_001_00
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_glitter_001_00_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_glitter_001_01
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_glitter_001_01_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_hitmewobble_001_00
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_hitmewobble_001_00_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_hitmewobble_001_01
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_hitmewobble_001_01_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_luco100bpm_002_00
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_luco100bpm_002_00_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_luco100bpm_002_01
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_luco100bpm_002_01_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_opdirt_002_00
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_opdirt_002_00_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_opdirt_002_01
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_opdirt_002_01_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_thisisit_001_00
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_thisisit_001_00_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_thisisit_001_01
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_thisisit_001_01_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_yougonnaregretit_002_00
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_yougonnaregretit_002_00_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_yougonnaregretit_002_01
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_yougonnaregretit_002_01_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_yourhead108bpm_001_00
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_yourhead108bpm_001_00_mirrored
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_yourhead108bpm_001_01
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_yourhead108bpm_001_01_mirrored
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_boogiewonderland_001_00
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_boogiewonderland_001_00_mirrored
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_boogiewonderland_001_01
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_boogiewonderland_001_01_mirrored
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_dennisscorpiocoffey_001_00
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_dennisscorpiocoffey_001_00_mirrored
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_dennisscorpiocoffey_001_01
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_dennisscorpiocoffey_001_01_mirrored
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_funk_001_00
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_funk_001_00_mirrored
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_funk_001_01
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_funk_001_01_mirrored
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_jamesbrownpapasgotabrandnewbag_001_00
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_jamesbrownpapasgotabrandnewbag_001_00_mirrored
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_jamesbrownpapasgotabrandnewbag_001_01
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_jamesbrownpapasgotabrandnewbag_001_01_mirrored
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_princesexydanser_001_00
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_princesexydanser_001_00_mirrored
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_princesexydanser_001_01
kthstreet_gLO_sFM_cAll_d01_mLO_ch01_princesexydanser_001_01_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO1_ch01_00
kthstreet_gPO_sFM_cAll_d01_mPO1_ch01_00_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO1_ch01_01
kthstreet_gPO_sFM_cAll_d01_mPO1_ch01_01_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO2_ch02_00
kthstreet_gPO_sFM_cAll_d01_mPO2_ch02_00_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO2_ch02_01
kthstreet_gPO_sFM_cAll_d01_mPO2_ch02_01_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO3_ch03_00
kthstreet_gPO_sFM_cAll_d01_mPO3_ch03_00_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO3_ch03_01
kthstreet_gPO_sFM_cAll_d01_mPO3_ch03_01_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO4_ch04_00
kthstreet_gPO_sFM_cAll_d01_mPO4_ch04_00_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO4_ch04_01
kthstreet_gPO_sFM_cAll_d01_mPO4_ch04_01_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO5_ch05_00
kthstreet_gPO_sFM_cAll_d01_mPO5_ch05_00_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO5_ch05_01
kthstreet_gPO_sFM_cAll_d01_mPO5_ch05_01_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO6_ch06_00
kthstreet_gPO_sFM_cAll_d01_mPO6_ch06_00_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO6_ch06_01
kthstreet_gPO_sFM_cAll_d01_mPO6_ch06_01_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO7_ch07_00
kthstreet_gPO_sFM_cAll_d01_mPO7_ch07_00_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO7_ch07_01
kthstreet_gPO_sFM_cAll_d01_mPO7_ch07_01_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_bbonthespot_003_00
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_bbonthespot_003_00_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_bbonthespot_003_01
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_bbonthespot_003_01_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_heropop_002_00
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_heropop_002_00_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_heropop_002_01
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_heropop_002_01_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_justus2_001_00
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_justus2_001_00_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_justus2_001_01
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_justus2_001_01_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_luco100bpm_001_00
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_luco100bpm_001_00_mirrored
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_luco100bpm_001_01
kthstreet_gPO_sFM_cAll_d01_mPO_ch01_luco100bpm_001_01_mirrored
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_atombounce_001_00
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_atombounce_001_00_mirrored
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_atombounce_001_01
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_atombounce_001_01_mirrored
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_bbonthespot_004_00
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_bbonthespot_004_00_mirrored
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_bbonthespot_004_01
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_bbonthespot_004_01_mirrored
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_funk_002_00
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_funk_002_00_mirrored
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_funk_002_01
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_funk_002_01_mirrored
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_justus2_002_00
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_justus2_002_00_mirrored
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_justus2_002_01
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_justus2_002_01_mirrored
================================================
FILE: data/motorica_dance/gen_files.txt
================================================
kthjazz_gCH_sFM_cAll_d02_mCH_ch01_whitemanpaulandhisorchestraloisiana_006_00
kthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_bennygoodmansugarfootstomp_003_00
kthmisc_gCA_sFM_cAll_d01_mCA_ch24_00
kthstreet_gKR_sFM_cAll_d01_mKR_ch01_chargedcableupyour_001_00
kthstreet_gLH_sFM_cAll_d02_mLH_ch01_lala_001_00
kthstreet_gLO_sFM_cAll_d02_mLO_ch01_arethafranklinrocksteady_002_00
kthstreet_gPO_sFM_cAll_d02_mPO_ch01_bombom_001_00
================================================
FILE: data/motorica_dance/pose_features.expmap.txt
================================================
RightFoot_alpha
RightFoot_beta
RightFoot_gamma
RightLeg_alpha
RightLeg_beta
RightLeg_gamma
RightUpLeg_alpha
RightUpLeg_beta
RightUpLeg_gamma
LeftFoot_alpha
LeftFoot_beta
LeftFoot_gamma
LeftLeg_alpha
LeftLeg_beta
LeftLeg_gamma
LeftUpLeg_alpha
LeftUpLeg_beta
LeftUpLeg_gamma
RightHand_alpha
RightHand_beta
RightHand_gamma
RightForeArm_alpha
RightForeArm_beta
RightForeArm_gamma
RightArm_alpha
RightArm_beta
RightArm_gamma
RightShoulder_alpha
RightShoulder_beta
RightShoulder_gamma
LeftHand_alpha
LeftHand_beta
LeftHand_gamma
LeftForeArm_alpha
LeftForeArm_beta
LeftForeArm_gamma
LeftArm_alpha
LeftArm_beta
LeftArm_gamma
LeftShoulder_alpha
LeftShoulder_beta
LeftShoulder_gamma
Head_alpha
Head_beta
Head_gamma
Neck_alpha
Neck_beta
Neck_gamma
Spine1_alpha
Spine1_beta
Spine1_gamma
Spine_alpha
Spine_beta
Spine_gamma
Hips_alpha
Hips_beta
Hips_gamma
Hips_Yposition
reference_dXposition
reference_dZposition
reference_dYrotation
================================================
FILE: experiments/dance_LDA-U.sh
================================================
#!/bin/bash
checkpoint=pretrained_models/dance_LDA-U.ckpt
dest_dir=results/generated/dance_LDA-U
if [ ! -d "${dest_dir}" ]; then
mkdir -p "${dest_dir}"
fi
data_dir=data/motorica_dance
wav_dir=data/motorica_dance
basenames=$(cat "${data_dir}/gen_files.txt")
start=0
seed=150
fps=30
trim_s=0
length_s=10
trim=$((trim_s*fps))
length=$((length_s*fps))
fixed_seed=false
gpu="cuda:0"
render_video=true
for wavfile in $basenames;
do
start=0
for postfix in 0 1 2 3 4 5 6 7 8 9 10 11
do
input_file=${wavfile}.audio29_${fps}fps.pkl
output_file=${wavfile::-3}_${postfix}_${style}
echo "start=${start}, len=${length}, postfix=${postfix}, seed=${seed}"
python synthesize.py --checkpoints="${checkpoint}" --data_dirs="${data_dir}" --input_files="${input_file}" --start=${start} --end=${length} --seed=${seed} --postfix=${postfix} --trim=${trim} --dest_dir=${dest_dir} --gpu=${gpu} --video=${render_video} --outfile=${output_file}
if [ "$fixed_seed" != "true" ]; then
seed=$((seed+1))
fi
echo seed=$seed
python utils/cut_wav.py ${wav_dir}/${wavfile::-3}.wav $(((start+trim)/fps)) $(((start+length-trim)/fps)) ${postfix} ${dest_dir}
if [ "$render_video" == "true" ]; then
ffmpeg -y -i ${dest_dir}/${output_file}.mp4 -i ${dest_dir}/${wavfile::-3}_${postfix}.wav ${dest_dir}/${output_file}_audio.mp4
rm ${dest_dir}/${output_file}.mp4
fi
start=$((start+length))
done
done
================================================
FILE: experiments/dance_LDA.sh
================================================
#!/bin/bash
checkpoint=pretrained_models/dance_LDA.ckpt
dest_dir=results/generated/dance_LDA
if [ ! -d "${dest_dir}" ]; then
mkdir -p "${dest_dir}"
fi
data_dir=data/motorica_dance
wav_dir=data/motorica_dance
basenames=$(cat "${data_dir}/gen_files.txt")
start=0
seed=150
fps=30
trim_s=0
length_s=10
trim=$((trim_s*fps))
length=$((length_s*fps))
fixed_seed=false
gpu="cuda:0"
render_video=true
for wavfile in $basenames;
do
start=0
style=$(echo $wavfile | awk -F "_" '{print $2}') #Coherent style parsed from file-name
for postfix in 0 1 2 3 4 5 6 7 8 9 10 11
do
input_file=${wavfile}.audio29_${fps}fps.pkl
output_file=${wavfile::-3}_${postfix}_${style}
echo "start=${start}, len=${length}, postfix=${postfix}, seed=${seed}"
python synthesize.py --checkpoints="${checkpoint}" --data_dirs="${data_dir}" --input_files="${input_file}" --styles="${style}" --start=${start} --end=${length} --seed=${seed} --postfix=${postfix} --trim=${trim} --dest_dir=${dest_dir} --gpu=${gpu} --video=${render_video} --outfile=${output_file}
if [ "$fixed_seed" != "true" ]; then
seed=$((seed+1))
fi
echo seed=$seed
python utils/cut_wav.py ${wav_dir}/${wavfile::-3}.wav $(((start+trim)/fps)) $(((start+length-trim)/fps)) ${postfix} ${dest_dir}
if [ "$render_video" == "true" ]; then
ffmpeg -y -i ${dest_dir}/${output_file}.mp4 -i ${dest_dir}/${wavfile::-3}_${postfix}.wav ${dest_dir}/${output_file}_audio.mp4
rm ${dest_dir}/${output_file}.mp4
fi
start=$((start+length))
done
done
================================================
FILE: experiments/dance_mix_experts.sh
================================================
#!/bin/bash
### Mixing N models/conditionings to generate a mix of styles. 1. Model A trained w.o. style conditioning 2. Model B with style S1
### 3. Model B with style S2. The sampling will be done with e=e0 + g1*(e1-e0) + g2*(e2-e0) + ...
### gn are guidance factors. E.g. g1=1, g2=0 => style S1 g1=0,g2=1 => S2, g1=1,g2=1 => mix of styles
# # Dance
checkpoint1=pretrained_models/dance_LDA-U.ckpt
checkpoint2=pretrained_models/dance_LDA.ckpt
checkpoint3=${checkpoint2}
dest_dir=results/dance_mix_experts
if [ ! -d "${dest_dir}" ]; then
mkdir -p "${dest_dir}"
fi
data_dir=data/motorica_dance
basenames=$(cat "${data_dir}/gen_files.txt")
# Different guidance factors for mixing models
guidance_factors_lst=("1.0,1.0" "0.5,0.5" "0.25,1.0" "1.0,0.25")
style=None,gJZ,gLO
start=0
seed=150
fps=30
trim_s=0
length_s=10
trim=$((trim_s*fps))
length=$((length_s*fps))
fixed_seed=false
gpu="cuda:0"
render_video=true
for wavfile in $basenames;
do
start=0
for postfix in 0
do
for guidance_factors in ${guidance_factors_lst[@]}; do
input_file=${wavfile}.audio29_${fps}fps.pkl
input_file2=${input_file}
input_file3=${input_file}
output_file=${wavfile::-3}_${postfix}_${style}_${guidance_factors}
echo Generating motion from ${input_file} to ${dest_dir}/${output_file}
echo "start=${start}, len=${length}, postfix=${postfix}, seed=${seed}"
python synthesize.py --checkpoints="${checkpoint1},${checkpoint2},${checkpoint3}" --data_dirs="${data_dir},${data_dir},${data_dir}" --input_files="${input_file},${input_file2},${input_file3}" --styles="${style}" --start=${start} --end=${length} --trim=${trim} --seed=${seed} --postfix=${postfix} --dest_dir=${dest_dir} --gf=${guidance_factors} --gpu=${gpu} --outfile=${output_file} --video=${render_video}
if [ "$fixed_seed" != "true" ]; then
seed=$((seed+1))
fi
echo seed=$seed
python utils/cut_wav.py ${data_dir}/${wavfile::-3}.wav $(((start+trim)/fps)) $(((start+length-trim)/fps)) ${postfix} ${dest_dir}
ffmpeg -y -i ${dest_dir}/${output_file}.mp4 -i ${dest_dir}/${wavfile::-3}_${postfix}.wav ${dest_dir}/${output_file}_audio.mp4
rm ${dest_dir}/${output_file}.mp4
start=$((start+length))
postfix=$((postfix+1))
done
done
done
================================================
FILE: hparams/diffusion_dance_LDA-U.yaml
================================================
Data:
segment_length: 150
style_index: [1]
trim_edges: 300
traindata_filename: dance_train_files_kth.txt
testdata_filename: dance_test_files_kth.txt
input_modality: audio35_30fps
output_modality: expmap_30fps
input_feats_file: ch0_spec_beatact_features.txt
datapipe_filename: data_pipe.expmap_30fps.sav
timestretch_prob: 0.2
timestretch_factor: 0.1
Diffusion:
name: tisa #tisa|conv
residual_layers: 20
residual_channels: 256
embedding_dim: 512
args:
tisa:
num_blocks: 2
num_heads: 8
activation: relu
dropout: 0.1
norm: LN
d_ff: 1024
seq_len: 150
use_v2: true
use_preln: false
bias: false
dilation_cycle: [0,1,2]
conv:
dilation_cycle_length: 10
unconditional: false
noise_schedule_start: 0.01
noise_schedule_end: 0.7
n_noise_schedule: 150
Infer:
eps: 1
seq_len: 25
Optim:
Schedule:
args:
lambda:
val: 10
multiplicative:
val: 10
step:
gamma: 0.99995
step_size: 10
name: step
warm_up: 3000
args:
adam:
betas:
- 0.9
- 0.999
eps: 1.0e-08
rmsprop:
eps: 1.0e-08
sgd:
momentum: 0.9
name: adam
Validation:
render: true
apply_dropout: false
render_every_n_epochs: 1
max_render_clips: 10
gen_synth_ctrl: false
lr: 0.0006
batch_size: 80
num_dataloader_workers: 1
pruning_amount: 0.0
quantization: false
Trainer:
accelerator: gpu
devices: [2]
accumulate_grad_batches: 1
default_root_dir: results/training/dance_LDA-U
gradient_clip_val: 25
deterministic: false
fast_dev_run: false
max_epochs: 10
min_epochs: 1
precision: 32
resume_from_checkpoint: null
================================================
FILE: hparams/diffusion_dance_LDA.yaml
================================================
Data:
segment_length: 150
style_index: [1]
trim_edges: 300
styles_file: dance_styles_kth.txt
traindata_filename: dance_train_files_kth.txt
testdata_filename: dance_test_files_kth.txt
input_modality: audio35_30fps
output_modality: expmap_30fps
input_feats_file: ch0_spec_beatact_features.txt
datapipe_filename: data_pipe.expmap_30fps.sav
timestretch_prob: 0.2
timestretch_factor: 0.1
Diffusion:
name: tisa #tisa|conv
residual_layers: 20
residual_channels: 256
embedding_dim: 512
args:
tisa:
num_blocks: 2
num_heads: 8
activation: relu
dropout: 0.1
norm: LN
d_ff: 1024
seq_len: 150
use_preln: false
bias: false
dilation_cycle: [0,1,2]
conv:
dilation_cycle_length: 10
unconditional: false
noise_schedule_start: 0.01
noise_schedule_end: 0.7
n_noise_schedule: 150
Infer:
eps: 1
seq_len: 25
Optim:
Schedule:
args:
lambda:
val: 10
multiplicative:
val: 10
step:
gamma: 0.99995
step_size: 10
name: step
warm_up: 3000
args:
adam:
betas:
- 0.9
- 0.999
eps: 1.0e-08
rmsprop:
eps: 1.0e-08
sgd:
momentum: 0.9
name: adam
Validation:
render: true
apply_dropout: false
render_every_n_epochs: 1
max_render_clips: 10
gen_synth_ctrl: false
lr: 0.0006
batch_size: 80
num_dataloader_workers: 1
pruning_amount: 0.0
quantization: false
Trainer:
accelerator: gpu
devices: [2]
accumulate_grad_batches: 1
default_root_dir: results/training/dance_LDA
gradient_clip_val: 25
deterministic: false
fast_dev_run: false
max_epochs: 10
min_epochs: 1
precision: 32
resume_from_checkpoint: null
================================================
FILE: models/BaseModel.py
================================================
# Copyright 2023 Motorica AB, Inc. All Rights Reserved.
import os
import torch
import numpy as np
from torch.optim import SGD, Adam, RMSprop
from torch.optim.lr_scheduler import LambdaLR, MultiplicativeLR, StepLR, CosineAnnealingWarmRestarts
from pytorch_lightning import LightningModule
from utils.logging_mixin import LoggingMixin
from typing import Tuple, Optional, Union, Dict
from argparse import Namespace
class BaseModel(LoggingMixin, LightningModule):
def __init__(self, conf: Optional[Union[Dict, Namespace]] = None, **kwargs):
super().__init__()
self.save_hyperparameters(conf)
scalers = self.hparams.Data["scalers"]
#Get input and output scalers from hparams
input_means=np.array([])
input_stds=np.array([])
if scalers["in_scaler"] is not None:
input_means = scalers["in_scaler"].mean_
input_stds = scalers["in_scaler"].scale_
self.input_means = torch.from_numpy(input_means)
self.input_scales = torch.from_numpy(input_stds)
self.output_means = torch.from_numpy(scalers["out_scaler"].mean_)
self.output_scales = torch.from_numpy(scalers["out_scaler"].scale_)
def get_scalers(self):
return self.hparams.Data["scalers"]
# standarize input
def standardizeInput(self, input_tensor):
return ((input_tensor - self.input_means.type_as(input_tensor)) / self.input_scales.type_as(input_tensor))
# standarize output
def standardizeOutput(self, output_tensor):
return ((output_tensor - self.output_means.type_as(output_tensor)) / self.output_scales.type_as(output_tensor))
# Add scale and means to output
def destandardizeInput(self, input_tensor):
return (input_tensor * self.input_scales.type_as(input_tensor) + self.input_means.type_as(input_tensor))
# Add scale and means to output
def destandardizeOutput(self, predictions):
return (predictions * self.output_scales.type_as(predictions) + self.output_means.type_as(predictions))
def configure_optimizers(self):
lr_params = self.hparams.Optim
optim_args = lr_params["args"][lr_params["name"]]
optimizers = {"adam": Adam, "sgd": SGD, "rmsprop": RMSprop}
# Define optimizer
optimizer = optimizers[lr_params["name"]](
self.parameters(), lr=self.hparams.lr, **optim_args
)
# Define Learning Rate Scheduling
def lambda1(val):
return lambda epoch: epoch // val
sched_params = self.hparams.Optim["Schedule"]
sched_name = sched_params["name"]
if not sched_name:
return optimizer
sched_args = sched_params["args"][sched_name]
if sched_name == "step":
scheduler = StepLR(optimizer, **sched_args)
elif sched_name == "multiplicative":
scheduler = MultiplicativeLR(
optimizer, lr_lambda=[lambda1(sched_args["val"])]
)
elif sched_name == "lambda":
scheduler = LambdaLR(optimizer, lr_lambda=[lambda1(sched_args["val"])])
elif sched_name == "cos_warm":
scheduler = CosineAnnealingWarmRestarts(optimizer, T_0=sched_args["T_0"])
else:
raise NotImplementedError("Unimplemented Scheduler!")
#return [optimizer], [scheduler]
return {
"optimizer": optimizer,
"lr_scheduler": {
"scheduler": scheduler,
"interval": "step",
},
}
# learning rate warm-up
def on_before_optimizer_step(self, optimizer, optimizer_idx):
lr = self.hparams.lr
#warm up lr
warm_up = self.hparams.Optim["Schedule"]["warm_up"]
if self.trainer.global_step < warm_up:
lr_scale = min(1.0, float(self.trainer.global_step + 1) / warm_up)
lr *= lr_scale
for pg in optimizer.param_groups:
pg["lr"] = lr
================================================
FILE: models/LightningModel.py
================================================
# Copyright 2023 Motorica AB, Inc. All Rights Reserved.
import os
import sys
import numpy as np
import torch
from torch import nn, Tensor
from torch.nn import functional as F
from models.BaseModel import BaseModel
from models.nn import LDA
from typing import Tuple, Optional, Union, Dict
from argparse import Namespace
class LitLDA(BaseModel):
def __init__(self, conf, **kwargs):
super().__init__(conf)
self.input_dim = 0
self.style_dim = 0
if self.hparams.Data["scalers"]["in_scaler"] is not None:
self.input_dim = self.hparams.Data["scalers"]["in_scaler"].mean_.shape[0]
if self.hparams.Data["scalers"]["style_scaler"] is not None:
self.style_dim = self.hparams.Data["scalers"]["style_scaler"].mean_.shape[0]
self.pose_dim = self.hparams.Data["scalers"]["out_scaler"].mean_.shape[0]
self.g_cond_dim = self.style_dim
self.unconditional = self.input_dim == 0
n_timesteps = self.hparams.Data["segment_length"]
diff_params = self.hparams.Diffusion
beta_min = diff_params["noise_schedule_start"]
beta_max = diff_params["noise_schedule_end"]
self.n_noise_schedule = diff_params["n_noise_schedule"]
self.noise_schedule_name = "linear"
self.noise_schedule = torch.linspace(beta_min, beta_max, self.n_noise_schedule)
self.noise_level = torch.cumprod(1 - self.noise_schedule, dim=0)
nn_name = diff_params["name"]
nn_args = diff_params["args"][nn_name]
self.diffusion_model = LDA(self.pose_dim,
self.hparams.Diffusion["residual_layers"],
self.hparams.Diffusion["residual_channels"],
self.hparams.Diffusion["embedding_dim"],
self.input_dim,
self.g_cond_dim,
self.n_noise_schedule,
nn_name,
nn_args)
self.loss_fn = nn.MSELoss()
def get_input_dim(self):
return self.input_dim
def get_style_dim(self):
return self.style_dim
def get_pose_dim(self):
return self.pose_dim
def diffusion(self, poses, t):
N, T, C = poses.shape
noise = torch.randn_like(poses)
noise_scale = self.noise_level.type_as(noise)[t].unsqueeze(1).unsqueeze(2).repeat(1,T,C)
noise_scale_sqrt = noise_scale**0.5
noisy_poses = noise_scale_sqrt * poses + (1.0 - noise_scale)**0.5 * noise
return noisy_poses, noise
def forward(self, batch):
ctrl, global_cond, poses =batch
N, T, C = poses.shape
num_noisesteps = self.n_noise_schedule
t = torch.randint(0, num_noisesteps, [N], device=poses.device)
noise = torch.randn_like(poses)
noise_scale = self.noise_level.type_as(noise)[t].unsqueeze(1).unsqueeze(2).repeat(1,T,C)
noise_scale_sqrt = noise_scale**0.5
noisy_poses = noise_scale_sqrt * poses + (1.0 - noise_scale)**0.5 * noise
noisy_poses, noise = self.diffusion(poses, t)
predicted = self.diffusion_model(noisy_poses, ctrl, global_cond, t)
loss = self.loss_fn(noise, predicted.squeeze(1))
return loss
def training_step(self, batch, batch_idx):
loss = self(batch)
self.log('Loss/train', loss, sync_dist=True)
return loss
def validation_step(self, batch, batch_idx):
loss = self(batch)
self.log('val_loss', loss, prog_bar=True, sync_dist=True)
if (self.trainer.global_step > 0 and
batch_idx==0 and
(self.trainer.current_epoch == self.trainer.max_epochs-1 or self.trainer.current_epoch % self.hparams.Validation["render_every_n_epochs"]==0)):
# Log results for the validation data
self.synthesize_and_log(batch, "val")
output = {"val_loss": loss}
return output
def validation_epoch_end(self, outputs):
avg_loss = torch.stack([x["val_loss"] for x in outputs]).mean()
self.log('Loss/val', avg_loss, sync_dist=True)
def synthesize_and_log(self, batch, log_prefix):
ctrl, g_cond, _ =batch
clips = self.synthesize(ctrl, g_cond)
self.log_jerk(clips[:,:,:self.pose_dim], log_prefix)
file_name = f"{self.current_epoch}_{self.global_step}_{log_prefix}"
self.log_results(clips.cpu().detach().numpy(), file_name, log_prefix, render_video=False)
def test_step(self, batch, batch_idx):
loss = self(batch)
self.synthesize_and_log(batch, "test")
output = {"test_loss": loss}
return output
def synthesize(self, ctrl, global_cond):
print("synthesize")
training_noise_schedule = self.noise_schedule.to(ctrl.device)
inference_noise_schedule = training_noise_schedule
talpha = 1 - training_noise_schedule
talpha_cum = torch.cumprod(talpha, dim=0)
beta = inference_noise_schedule
alpha = 1 - beta
alpha_cum = torch.cumprod(alpha, dim=0)
T = []
for s in range(len(inference_noise_schedule)):
for t in range(len(training_noise_schedule) - 1):
if talpha_cum[t+1] <= alpha_cum[s] <= talpha_cum[t]:
twiddle = (talpha_cum[t]**0.5 - alpha_cum[s]**0.5) / (talpha_cum[t]**0.5 - talpha_cum[t+1]**0.5)
T.append(t + twiddle)
break
if len(ctrl.shape) == 2:# Expand rank 2 tensors by adding a batch dimension.
ctrl = ctrl.unsqueeze(0)
global_cond = global_cond.unsqueeze(0)
poses = torch.randn(ctrl.shape[0], ctrl.shape[1], self.pose_dim, device=ctrl.device)
nbatch = poses.size(0)
noise_scale = (alpha_cum**0.5).type_as(poses).unsqueeze(1)
for n in range(len(alpha) - 1, -1, -1):
c1 = 1 / alpha[n]**0.5
c2 = beta[n] / (1 - alpha_cum[n])**0.5
poses = c1 * (poses - c2 * self.diffusion_model(poses, ctrl, global_cond, T[n].unsqueeze(-1)).squeeze(1))
if n > 0:
noise = torch.randn_like(poses)
sigma = ((1.0 - alpha_cum[n-1]) / (1.0 - alpha_cum[n]) * beta[n])**0.5
poses += sigma * noise
anim_clip = self.destandardizeOutput(poses)
if not self.unconditional:
out_ctrl = self.destandardizeInput(ctrl)
anim_clip = torch.cat((anim_clip, out_ctrl), dim=2)
return anim_clip
================================================
FILE: models/nn.py
================================================
# Copyright 2023 Motorica AB, Inc. All Rights Reserved.
import torch
import torch.nn as nn
import torch.nn.functional as F
from models.transformer.tisa_transformer import TisaTransformer
from math import sqrt
class Conv1dLayer(nn.Module):
def __init__(self, in_channels, out_channels, kernel_size, padding=0, dilation=1):
super().__init__()
self.conv1d = nn.Conv1d(in_channels, out_channels, kernel_size=kernel_size, padding=padding, dilation=dilation)
nn.init.kaiming_normal_(self.conv1d.weight)
def forward(self, x):
return self.conv1d(x.permute(0,2,1)).permute(0,2,1)
def silu(x):
return x * torch.sigmoid(x)
class DiffusionEmbedding(nn.Module):
def __init__(self, max_steps, in_channels, hidden_channels):
super().__init__()
self.in_channels = in_channels
self.register_buffer('embedding', self._build_embedding(max_steps), persistent=False)
self.projection1 = nn.Linear(in_channels, hidden_channels)
self.projection2 = nn.Linear(hidden_channels, hidden_channels)
def forward(self, diffusion_step):
if diffusion_step.dtype in [torch.int32, torch.int64]:
x = self.embedding[diffusion_step]
else:
x = self._lerp_embedding(diffusion_step)
x = self.projection1(x)
x = silu(x)
x = self.projection2(x)
x = silu(x)
return x
def _lerp_embedding(self, t):
low_idx = torch.floor(t).long()
high_idx = torch.ceil(t).long()
low = self.embedding[low_idx]
high = self.embedding[high_idx]
return low + (high - low) * (t - low_idx)
def _build_embedding(self, max_steps):
steps = torch.arange(max_steps).unsqueeze(1) # [T,1]
dims = torch.arange(64).unsqueeze(0) # [1,64]
table = steps * 10.0**(dims * 4.0 / 63.0) # [T,64]
table = torch.cat([torch.sin(table), torch.cos(table)], dim=1)
return table
class ResidualBlock(nn.Module):
def __init__(self, residual_channels,
embedding_dim,
l_cond_dim,
nn_name,
nn_args,
index):
super().__init__()
if nn_name=="tisa":
dilation_cycle = nn_args["dilation_cycle"]
dilation=dilation_cycle[(index % len(dilation_cycle))]
self.nn = TisaTransformer(residual_channels, 2 * residual_channels, d_model=residual_channels, num_blocks=nn_args["num_blocks"], num_heads=nn_args["num_heads"], activation=nn_args["activation"], norm=nn_args["norm"], drop_prob=nn_args["dropout"], d_ff=nn_args["d_ff"], seqlen=nn_args["seq_len"], use_preln=nn_args["use_preln"], bias=nn_args["bias"], dilation=dilation)
elif nn_name=="conv":
dilation=2**(index % nn_args["dilation_cycle_length"])
self.nn = Conv1dLayer(residual_channels, 2 * residual_channels, 3, padding=dilation, dilation=dilation)
else:
raise ValueError(f"Unknown nn_name: {nn_name}")
self.l_cond_dim = l_cond_dim
self.diffusion_projection = nn.Linear(embedding_dim, residual_channels)
self.local_cond_projection = nn.Linear(l_cond_dim, residual_channels)
self.output_projection = Conv1dLayer(residual_channels, 2 * residual_channels, 1)
self.residual_channels = residual_channels
def forward(self, x, diffusion_step, local_cond):
diffusion_step = self.diffusion_projection(diffusion_step).unsqueeze(1)
y = x + diffusion_step
if self.l_cond_dim > 0:
y += self.local_cond_projection(local_cond)
y = self.nn(y).squeeze(-1)
gate, filter = torch.chunk(y, 2, dim=2)
y = torch.sigmoid(gate) * torch.tanh(filter)
y = self.output_projection(y)
residual, skip = torch.chunk(y, 2, dim=2)
return (x + residual) / sqrt(2.0), skip
class LDA(nn.Module):
def __init__(self,
pose_dim,
residual_layers,
residual_channels,
embedding_dim,
l_cond_dim,
g_cond_dim,
n_noise_schedule,
nn_name,
nn_args):
super().__init__()
self.input_projection = Conv1dLayer(pose_dim, residual_channels, 1)
self.diffusion_embedding = DiffusionEmbedding(n_noise_schedule, 128, embedding_dim)
self.residual_layers = nn.ModuleList([
ResidualBlock(residual_channels,
embedding_dim,
l_cond_dim + g_cond_dim,
nn_name,
nn_args,
i)
for i in range(residual_layers)
])
self.skip_projection = Conv1dLayer(residual_channels, residual_channels, 1)
self.output_projection = Conv1dLayer(residual_channels, pose_dim, 1)
nn.init.zeros_(self.output_projection.conv1d.weight)
self.l_cond_dim = l_cond_dim
self.g_cond_dim = g_cond_dim
def forward(self, x, local_cond, global_cond, diffusion_step):
x = self.input_projection(x)
x = F.relu(x)
diffusion_step = self.diffusion_embedding(diffusion_step)
if self.g_cond_dim > 0:
local_cond=torch.cat((local_cond, global_cond), dim=2)
skip = None
i=1
for layer in self.residual_layers:
x, skip_connection = layer(x, diffusion_step, local_cond)
skip = skip_connection if skip is None else skip_connection + skip
if skip is not None:
x = skip / sqrt(len(self.residual_layers))
x = self.skip_projection(x)
x = F.relu(x)
x = self.output_projection(x)
return x
================================================
FILE: models/transformer/tisa_transformer.py
================================================
# Copyright 2023 Motorica AB, Inc. All Rights Reserved.
import torch
import torch.nn as nn
import torch.nn.functional as F
from models.transformer.tisa_v2 import TisaV2
class TisaTransformer(nn.Module):
def __init__(
self,
in_channels,
out_channels,
d_model,
num_blocks,
num_heads,
activation,
norm,
drop_prob,
d_ff=2048,
tisa_num_kernels=21,
seqlen=128,
use_preln=False,
bias=False,
dilation=1
):
super(TisaTransformer, self).__init__()
self.in_proj = nn.Linear(in_channels, d_model)
self.attention_blocks = nn.ModuleList(
[
AttnBlock(d_model, num_heads, activation, norm, drop_prob, tisa_num_kernels, seqlen, use_preln, d_ff, bias=bias, dilation=dilation)
for _ in range(num_blocks)
]
)
self.out_proj = nn.Linear(d_model, out_channels)
def forward(self, x):
x = self.in_proj(x)
for layer in self.attention_blocks:
x = layer(x)
x = self.out_proj(x)
return x
class Norm(nn.Module):
def __init__(self, d_model, eps = 1e-6):
super().__init__()
self.size = d_model
# create two learnable parameters to calibrate normalisation
self.alpha = nn.Parameter(torch.ones(self.size))
self.bias = nn.Parameter(torch.zeros(self.size))
self.eps = eps
def forward(self, x):
norm = self.alpha * (x - x.mean(dim=-1, keepdim=True)) \
/ (x.std(dim=-1, keepdim=True) + self.eps) + self.bias
return norm
class AttnBlock(nn.Module):
def __init__(self, d_model, num_heads, activation, norm, drop_prob, tisa_num_kernels, seqlen, use_preln=False, d_ff=2048, bias=False, dilation=1):
super(AttnBlock, self).__init__()
self.use_preln = use_preln
self.attn = GatedAttn(d_model, num_heads=num_heads, activation=activation, seqlen=seqlen, drop_prob=drop_prob, tisa_num_kernels=tisa_num_kernels)
if (dilation>0):
self.ff = ConvLayer(d_model, d_ff, activation=activation, dilation=dilation, dropout=drop_prob, bias=bias)
else:
self.ff = FeedForward(d_model, d_ff, activation=activation, dropout=drop_prob, bias=bias)
if norm == "T5":
self.norm_1 = T5LayerNorm(d_model)
self.norm_2 = T5LayerNorm(d_model)
elif norm == "LN":
self.norm_1 = nn.LayerNorm(d_model)
self.norm_2 = nn.LayerNorm(d_model)
else:
raise ValueError(f"unknown norm: {norm}")
self.dropout_1 = nn.Dropout(drop_prob)
self.dropout_2 = nn.Dropout(drop_prob)
def forward(self, x):
if self.use_preln:
x = self.dropout_1(self.attn(self.norm_2(x))) + x
else:
x = self.dropout_1(self.attn(x)) + x
x = self.norm_2(x)
if self.use_preln:
x = self.dropout_2(self.ff(self.norm_1(x))) + x
else:
x = self.dropout_2(self.ff(x)) + x
x = self.norm_1(x)
return x
class T5LayerNorm(nn.Module):
def __init__(self, hidden_size, eps=1e-6):
"""
Construct a layernorm module in the T5 style. No bias and no subtraction of mean.
"""
super().__init__()
self.weight = nn.Parameter(torch.ones(hidden_size))
self.variance_epsilon = eps
def forward(self, hidden_states):
# T5 uses a layer_norm which only scales and doesn't shift, which is also known as Root Mean
# Square Layer Normalization https://arxiv.org/abs/1910.07467 thus varience is calculated
# w/o mean and there is no bias. Additionally we want to make sure that the accumulation for
# half-precision inputs is done in fp32
variance = hidden_states.to(torch.float32).pow(2).mean(-1, keepdim=True)
hidden_states = hidden_states * torch.rsqrt(variance + self.variance_epsilon)
return self.weight * hidden_states
class ConvLayer(nn.Module):
def __init__(self, d_model, d_ff=2048, activation="relu", dilation=1, dropout = 0.1, bias=False):
super().__init__()
# We set d_ff as a default to 2048
self.conv_1 = nn.Conv1d(d_model, d_ff, 3, padding=dilation, dilation=dilation, bias=bias)
self.dropout = nn.Dropout(dropout)
self.conv_2 = nn.Conv1d(d_ff, d_model, 3, padding=dilation, dilation=dilation, bias=bias)
if activation=="relu":
self.act = nn.ReLU()
elif activation=="gelu":
self.act = nn.GELU()
def forward(self, x):
x = self.dropout(self.act(self.conv_1(x.permute(0,2,1))))
x = self.conv_2(x)
return x.permute(0,2,1)
class FeedForward(nn.Module):
def __init__(self, d_model, d_ff=2048, activation="RELU", dropout = 0.1, bias=False):
super().__init__()
# We set d_ff as a default to 2048
self.linear_1 = nn.Linear(d_model, d_ff, bias=bias)
self.dropout = nn.Dropout(dropout)
self.linear_2 = nn.Linear(d_ff, d_model, bias=bias)
if activation=="relu":
self.act = nn.ReLU()
elif activation=="gelu":
self.act = nn.GELU()
def forward(self, x):
x = self.dropout(self.act(self.linear_1(x)))
x = self.linear_2(x)
return x
class GatedAttn(nn.Module):
"""Gated Multi-Head Self-Attention Block
Based on the paper:
"Attention Is All You Need"
by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones,
Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin
(https://arxiv.org/abs/1706.03762).
Args:
d_model (int): Number of channels in the input.
num_heads (int): Number of attention heads.
drop_prob (float): Dropout probability.
"""
def __init__(self, d_model, num_heads=4, activation="RELU", seqlen=128, drop_prob=0.0, tisa_num_kernels=21):
super(GatedAttn, self).__init__()
assert d_model%num_heads==0, f"num_heads ({num_heads}) is not evenly divisible by d_model ({d_model})"
self.d_model = d_model
self.num_heads = num_heads
if drop_prob>0:
self.dropout = nn.Dropout(drop_prob)
else:
self.dropout = None
self.in_proj = nn.Linear(d_model, 3 * d_model, bias=False)
self.gate = nn.Linear(d_model, 2 * d_model)
self.key_depth_per_head = torch.tensor(self.d_model / self.num_heads)
self.position_scorer = TisaV2(self.num_heads,
tisa_num_kernels=tisa_num_kernels,
tisa_dropout_prob=drop_prob,
num_position_agnostic_heads=0,
max_field_view=seqlen//2,
min_field_view=5)
self.position_scorer._init_weights()
def forward(self, x):
b, h, c = x.size()
_, seq_len, num_channels = x.size()
# Compute q, k, v
memory, query = torch.split(self.in_proj(x), (2 * c, c), dim=-1)
q = self.split_last_dim(query, self.num_heads)
k, v = [
self.split_last_dim(tensor, self.num_heads)
for tensor in torch.split(memory, self.d_model, dim=2)
]
# Compute attention and reshape
x = self.dot_product_attention(q, k, v)
x = self.combine_last_two_dim(x.permute(0, 2, 1, 3))
x = self.gate(x)
a, b = x.chunk(2, dim=-1)
x = a * torch.sigmoid(b)
return x
def dot_product_attention(self, q, k, v):
"""Dot-product attention.
Args:
q (torch.Tensor): Queries of shape (batch, heads, length_q, depth_k)
k (torch.Tensor): Keys of shape (batch, heads, length_kv, depth_k)
v (torch.Tensor): Values of shape (batch, heads, length_kv, depth_v)
bias (bool): Use bias for attention.
Returns:
attn (torch.Tensor): Output of attention mechanism.
"""
weights = torch.matmul(q, k.permute(0, 1, 3, 2))
weights /= torch.sqrt(self.key_depth_per_head)
seq_len = weights.shape[-1]
weights += self.position_scorer(seq_len)
weights = F.softmax(weights, dim=-1)
if self.dropout is not None:
weights = self.dropout(weights)
attn = torch.matmul(weights, v)
return attn
@staticmethod
def split_last_dim(x, n):
"""Reshape x so that the last dimension becomes two dimensions.
The first of these two dimensions is n.
Args:
x (torch.Tensor): Tensor with shape (..., m)
n (int): Size of second-to-last dimension.
Returns:
ret (torch.Tensor): Resulting tensor with shape (..., n, m/n)
"""
#import pdb;pdb.set_trace()
old_shape = list(x.size())
last = old_shape[-1]
if last is not None:
new_shape = old_shape[:-1] + [n] + [last // n]
else:
new_shape = old_shape[:-1] + [n]
ret = x.view(new_shape)
return ret.permute(0, 2, 1, 3)
@staticmethod
def combine_last_two_dim(x):
"""Merge the last two dimensions of `x`.
Args:
x (torch.Tensor): Tensor with shape (..., m, n)
Returns:
ret (torch.Tensor): Resulting tensor with shape (..., m * n)
"""
old_shape = list(x.size())
a, b = old_shape[-2:]
new_shape = old_shape[:-2] + [a * b]
ret = x.contiguous().view(new_shape)
return ret
================================================
FILE: models/transformer/tisa_v2.py
================================================
# Copyright 2023 Ulme Wennberg, Inc. All Rights Reserved.
import torch
from torch import nn
from typing import List
import math
class TisaV2(nn.Module):
def __init__(self,
num_attention_heads,
tisa_num_kernels,
tisa_dropout_prob,
num_position_agnostic_heads,
max_field_view,
min_field_view,
p_eps = 1e-8):
super().__init__()
self.num_attention_heads = num_attention_heads
self.num_kernels = tisa_num_kernels
self.tisa_dropout_prob = tisa_dropout_prob
self.num_position_agnostic_heads = num_position_agnostic_heads
self.max_field_view = max_field_view
self.min_field_view = min_field_view
self.p_eps = p_eps
self.eps = 1e-8
self.offsets = nn.Parameter(
torch.zeros(1, self.num_kernels, self.num_attention_heads, 1, 1)
)
self.amplitudes = nn.Parameter(
torch.zeros(1, self.num_kernels, self.num_attention_heads, 1, 1)
)
self.sharpness = nn.Parameter(
torch.zeros(1, self.num_kernels, self.num_attention_heads, 1, 1)
)
self.bias = nn.Parameter(torch.zeros(1, self.num_attention_heads, 1, 1))
self.dropout = nn.Dropout(self.tisa_dropout_prob)
self.num_position_agnostic_heads = self.num_position_agnostic_heads
self.num_position_aware_heads = (
self.num_attention_heads - self.num_position_agnostic_heads
)
self.position_agnostic_heads = torch.arange(
self.num_attention_heads - self.num_position_agnostic_heads,
self.num_attention_heads,
)
assert 0 < self.p_eps < 1
"""
exp ( - field_view * m) = p_eps
- log(p_eps) = field_view * m
m = - log(p_eps) / field_view
"""
self.one_side_min_field_view = self.min_field_view / 2
self.one_side_max_field_view = self.max_field_view / 2
self.first_slope = -math.log(self.p_eps) / self.one_side_min_field_view
self.last_slope = -math.log(self.p_eps) / self.one_side_max_field_view
self.slopes = nn.Parameter(
(
self.first_slope
* (self.last_slope / self.first_slope)
** (
torch.arange(self.num_attention_heads)
/ (self.num_position_aware_heads - 1)
)
).reshape(self.num_attention_heads, 1, 1),
requires_grad=False,
)
# Disable exponential decay for position agnostic heads
self.slopes[self.position_agnostic_heads, 0, 0] = 0.0
def create_relative_offsets(self, seq_len):
"""Creates offsets for all the relative distances between
-seq_len + 1 to seq_len - 1."""
return (
torch.arange(-seq_len, seq_len + 1, device=self.offsets.device)
.unsqueeze(0)
.unsqueeze(0)
)
def forward(self, dim=-1, skip_apply_dropout=False):
"""Computes the translation-invariant positional contribution to the
attention matrix in the self-attention module of transformer models."""
indices_from = torch.arange(dim, device=self.offsets.device).unsqueeze(0).unsqueeze(0)
indices_to = indices_from
if not self.num_kernels:
return torch.zeros(
(self.num_attention_heads, indices_from.shape[-1], indices_to.shape[-1])
)
params = (indices_to.unsqueeze(-1) - indices_from.unsqueeze(-2)).unsqueeze(-4)
exponential_decay_arguments = -params.abs() * self.slopes
params = params.unsqueeze(-4) - self.offsets
params = params / self.sharpness
params = self.amplitudes.abs() * torch.sigmoid(self.amplitudes.sign() * params)
if self.training and not skip_apply_dropout:
params = self.dropout(params)
params = params.sum(dim=-4)
# Make final dimensions completely position agnostic
params[:, :, self.position_agnostic_heads] = 0.0
params += self.eps + self.bias.abs()
params = torch.log(params)
params = params + exponential_decay_arguments
return params.squeeze(0)
def _init_weights(self):
"""Initialize the weights"""
torch.nn.init.normal_(self.offsets, mean=0.0, std=15.0)
torch.nn.init.normal_(self.amplitudes, mean=0.0, std=0.01)
self.sharpness.data.fill_(5.0)
self.bias.data.fill_(1.0)
================================================
FILE: pretrained_models/.gitkeep
================================================
================================================
FILE: pretrained_models/README.md
================================================
Please [download our pretrained dance models here](https://zenodo.org/record/8156769) and place them in this folder.
================================================
FILE: pymo/Pivots.py
================================================
import numpy as np
from pymo.Quaternions import Quaternions
class Pivots:
"""
Pivots is an ndarray of angular rotations
This wrapper provides some functions for
working with pivots.
These are particularly useful as a number
of atomic operations (such as adding or
subtracting) cannot be achieved using
the standard arithmatic and need to be
defined differently to work correctly
"""
def __init__(self, ps): self.ps = np.array(ps)
def __str__(self): return "Pivots("+ str(self.ps) + ")"
def __repr__(self): return "Pivots("+ repr(self.ps) + ")"
def __add__(self, other): return Pivots(np.arctan2(np.sin(self.ps + other.ps), np.cos(self.ps + other.ps)))
def __sub__(self, other): return Pivots(np.arctan2(np.sin(self.ps - other.ps), np.cos(self.ps - other.ps)))
def __mul__(self, other): return Pivots(self.ps * other.ps)
def __div__(self, other): return Pivots(self.ps / other.ps)
def __mod__(self, other): return Pivots(self.ps % other.ps)
def __pow__(self, other): return Pivots(self.ps ** other.ps)
def __lt__(self, other): return self.ps < other.ps
def __le__(self, other): return self.ps <= other.ps
def __eq__(self, other): return self.ps == other.ps
def __ne__(self, other): return self.ps != other.ps
def __ge__(self, other): return self.ps >= other.ps
def __gt__(self, other): return self.ps > other.ps
def __abs__(self): return Pivots(abs(self.ps))
def __neg__(self): return Pivots(-self.ps)
def __iter__(self): return iter(self.ps)
def __len__(self): return len(self.ps)
def __getitem__(self, k): return Pivots(self.ps[k])
def __setitem__(self, k, v): self.ps[k] = v.ps
def _ellipsis(self): return tuple(map(lambda x: slice(None), self.shape))
def quaternions(self, plane='xz'):
fa = self._ellipsis()
axises = np.ones(self.ps.shape + (3,))
axises[fa + ("xyz".index(plane[0]),)] = 0.0
axises[fa + ("xyz".index(plane[1]),)] = 0.0
return Quaternions.from_angle_axis(self.ps, axises)
def directions(self, plane='xz'):
dirs = np.zeros((len(self.ps), 3))
dirs["xyz".index(plane[0])] = np.sin(self.ps)
dirs["xyz".index(plane[1])] = np.cos(self.ps)
return dirs
def normalized(self):
xs = np.copy(self.ps)
while np.any(xs > np.pi): xs[xs > np.pi] = xs[xs > np.pi] - 2 * np.pi
while np.any(xs < -np.pi): xs[xs < -np.pi] = xs[xs < -np.pi] + 2 * np.pi
return Pivots(xs)
def interpolate(self, ws):
dir = np.average(self.directions, weights=ws, axis=0)
return np.arctan2(dir[2], dir[0])
def copy(self):
return Pivots(np.copy(self.ps))
@property
def shape(self):
return self.ps.shape
@classmethod
def from_quaternions(cls, qs, forward='z', plane='xz'):
ds = np.zeros(qs.shape + (3,))
ds[...,'xyz'.index(forward)] = 1.0
return Pivots.from_directions(qs * ds, plane=plane)
@classmethod
def from_directions(cls, ds, plane='xz'):
ys = ds[...,'xyz'.index(plane[0])]
xs = ds[...,'xyz'.index(plane[1])]
return Pivots(np.arctan2(ys, xs))
================================================
FILE: pymo/Quaternions.py
================================================
import numpy as np
class Quaternions:
"""
Quaternions is a wrapper around a numpy ndarray
that allows it to act as if it were an narray of
a quaternion data type.
Therefore addition, subtraction, multiplication,
division, negation, absolute, are all defined
in terms of quaternion operations such as quaternion
multiplication.
This allows for much neater code and many routines
which conceptually do the same thing to be written
in the same way for point data and for rotation data.
The Quaternions class has been desgined such that it
should support broadcasting and slicing in all of the
usual ways.
"""
def __init__(self, qs):
if isinstance(qs, np.ndarray):
if len(qs.shape) == 1: qs = np.array([qs])
self.qs = qs
return
if isinstance(qs, Quaternions):
self.qs = qs.qs
return
raise TypeError('Quaternions must be constructed from iterable, numpy array, or Quaternions, not %s' % type(qs))
def __str__(self): return "Quaternions("+ str(self.qs) + ")"
def __repr__(self): return "Quaternions("+ repr(self.qs) + ")"
""" Helper Methods for Broadcasting and Data extraction """
@classmethod
def _broadcast(cls, sqs, oqs, scalar=False):
if isinstance(oqs, float): return sqs, oqs * np.ones(sqs.shape[:-1])
ss = np.array(sqs.shape) if not scalar else np.array(sqs.shape[:-1])
os = np.array(oqs.shape)
if len(ss) != len(os):
raise TypeError('Quaternions cannot broadcast together shapes %s and %s' % (sqs.shape, oqs.shape))
if np.all(ss == os): return sqs, oqs
if not np.all((ss == os) | (os == np.ones(len(os))) | (ss == np.ones(len(ss)))):
raise TypeError('Quaternions cannot broadcast together shapes %s and %s' % (sqs.shape, oqs.shape))
sqsn, oqsn = sqs.copy(), oqs.copy()
for a in np.where(ss == 1)[0]: sqsn = sqsn.repeat(os[a], axis=a)
for a in np.where(os == 1)[0]: oqsn = oqsn.repeat(ss[a], axis=a)
return sqsn, oqsn
""" Adding Quaterions is just Defined as Multiplication """
def __add__(self, other): return self * other
def __sub__(self, other): return self / other
""" Quaterion Multiplication """
def __mul__(self, other):
"""
Quaternion multiplication has three main methods.
When multiplying a Quaternions array by Quaternions
normal quaternion multiplication is performed.
When multiplying a Quaternions array by a vector
array of the same shape, where the last axis is 3,
it is assumed to be a Quaternion by 3D-Vector
multiplication and the 3D-Vectors are rotated
in space by the Quaternions.
When multipplying a Quaternions array by a scalar
or vector of different shape it is assumed to be
a Quaternions by Scalars multiplication and the
Quaternions are scaled using Slerp and the identity
quaternions.
"""
""" If Quaternions type do Quaternions * Quaternions """
if isinstance(other, Quaternions):
sqs, oqs = Quaternions._broadcast(self.qs, other.qs)
q0 = sqs[...,0]; q1 = sqs[...,1];
q2 = sqs[...,2]; q3 = sqs[...,3];
r0 = oqs[...,0]; r1 = oqs[...,1];
r2 = oqs[...,2]; r3 = oqs[...,3];
qs = np.empty(sqs.shape)
qs[...,0] = r0 * q0 - r1 * q1 - r2 * q2 - r3 * q3
qs[...,1] = r0 * q1 + r1 * q0 - r2 * q3 + r3 * q2
qs[...,2] = r0 * q2 + r1 * q3 + r2 * q0 - r3 * q1
qs[...,3] = r0 * q3 - r1 * q2 + r2 * q1 + r3 * q0
return Quaternions(qs)
""" If array type do Quaternions * Vectors """
if isinstance(other, np.ndarray) and other.shape[-1] == 3:
vs = Quaternions(np.concatenate([np.zeros(other.shape[:-1] + (1,)), other], axis=-1))
return (self * (vs * -self)).imaginaries
""" If float do Quaternions * Scalars """
if isinstance(other, np.ndarray) or isinstance(other, float):
return Quaternions.slerp(Quaternions.id_like(self), self, other)
raise TypeError('Cannot multiply/add Quaternions with type %s' % str(type(other)))
def __div__(self, other):
"""
When a Quaternion type is supplied, division is defined
as multiplication by the inverse of that Quaternion.
When a scalar or vector is supplied it is defined
as multiplicaion of one over the supplied value.
Essentially a scaling.
"""
if isinstance(other, Quaternions): return self * (-other)
if isinstance(other, np.ndarray): return self * (1.0 / other)
if isinstance(other, float): return self * (1.0 / other)
raise TypeError('Cannot divide/subtract Quaternions with type %s' + str(type(other)))
def __eq__(self, other): return self.qs == other.qs
def __ne__(self, other): return self.qs != other.qs
def __neg__(self):
""" Invert Quaternions """
return Quaternions(self.qs * np.array([[1, -1, -1, -1]]))
def __abs__(self):
""" Unify Quaternions To Single Pole """
qabs = self.normalized().copy()
top = np.sum(( qabs.qs) * np.array([1,0,0,0]), axis=-1)
bot = np.sum((-qabs.qs) * np.array([1,0,0,0]), axis=-1)
qabs.qs[top < bot] = -qabs.qs[top < bot]
return qabs
def __iter__(self): return iter(self.qs)
def __len__(self): return len(self.qs)
def __getitem__(self, k): return Quaternions(self.qs[k])
def __setitem__(self, k, v): self.qs[k] = v.qs
@property
def lengths(self):
return np.sum(self.qs**2.0, axis=-1)**0.5
@property
def reals(self):
return self.qs[...,0]
@property
def imaginaries(self):
return self.qs[...,1:4]
@property
def shape(self): return self.qs.shape[:-1]
def repeat(self, n, **kwargs):
return Quaternions(self.qs.repeat(n, **kwargs))
def normalized(self):
return Quaternions(self.qs / self.lengths[...,np.newaxis])
def log(self):
norm = abs(self.normalized())
imgs = norm.imaginaries
lens = np.sqrt(np.sum(imgs**2, axis=-1))
lens = np.arctan2(lens, norm.reals) / (lens + 1e-10)
return imgs * lens[...,np.newaxis]
def constrained(self, axis):
rl = self.reals
im = np.sum(axis * self.imaginaries, axis=-1)
t1 = -2 * np.arctan2(rl, im) + np.pi
t2 = -2 * np.arctan2(rl, im) - np.pi
top = Quaternions.exp(axis[np.newaxis] * (t1[:,np.newaxis] / 2.0))
bot = Quaternions.exp(axis[np.newaxis] * (t2[:,np.newaxis] / 2.0))
img = self.dot(top) > self.dot(bot)
ret = top.copy()
ret[ img] = top[ img]
ret[~img] = bot[~img]
return ret
def constrained_x(self): return self.constrained(np.array([1,0,0]))
def constrained_y(self): return self.constrained(np.array([0,1,0]))
def constrained_z(self): return self.constrained(np.array([0,0,1]))
def dot(self, q): return np.sum(self.qs * q.qs, axis=-1)
def copy(self): return Quaternions(np.copy(self.qs))
def reshape(self, s):
self.qs.reshape(s)
return self
def interpolate(self, ws):
return Quaternions.exp(np.average(abs(self).log, axis=0, weights=ws))
def euler(self, order='xyz'):
q = self.normalized().qs
q0 = q[...,0]
q1 = q[...,1]
q2 = q[...,2]
q3 = q[...,3]
es = np.zeros(self.shape + (3,))
if order == 'xyz':
es[...,0] = np.arctan2(2 * (q0 * q1 + q2 * q3), 1 - 2 * (q1 * q1 + q2 * q2))
es[...,1] = np.arcsin((2 * (q0 * q2 - q3 * q1)).clip(-1,1))
es[...,2] = np.arctan2(2 * (q0 * q3 + q1 * q2), 1 - 2 * (q2 * q2 + q3 * q3))
elif order == 'yzx':
es[...,0] = np.arctan2(2 * (q1 * q0 - q2 * q3), -q1 * q1 + q2 * q2 - q3 * q3 + q0 * q0)
es[...,1] = np.arctan2(2 * (q2 * q0 - q1 * q3), q1 * q1 - q2 * q2 - q3 * q3 + q0 * q0)
es[...,2] = np.arcsin((2 * (q1 * q2 + q3 * q0)).clip(-1,1))
else:
raise NotImplementedError('Cannot convert from ordering %s' % order)
"""
# These conversion don't appear to work correctly for Maya.
# http://bediyap.com/programming/convert-quaternion-to-euler-rotations/
if order == 'xyz':
es[...,0] = np.arctan2(2 * (q0 * q3 - q1 * q2), q0 * q0 + q1 * q1 - q2 * q2 - q3 * q3)
es[...,1] = np.arcsin((2 * (q1 * q3 + q0 * q2)).clip(-1,1))
es[...,2] = np.arctan2(2 * (q0 * q1 - q2 * q3), q0 * q0 - q1 * q1 - q2 * q2 + q3 * q3)
elif order == 'yzx':
es[...,0] = np.arctan2(2 * (q0 * q1 - q2 * q3), q0 * q0 - q1 * q1 + q2 * q2 - q3 * q3)
es[...,1] = np.arcsin((2 * (q1 * q2 + q0 * q3)).clip(-1,1))
es[...,2] = np.arctan2(2 * (q0 * q2 - q1 * q3), q0 * q0 + q1 * q1 - q2 * q2 - q3 * q3)
elif order == 'zxy':
es[...,0] = np.arctan2(2 * (q0 * q2 - q1 * q3), q0 * q0 - q1 * q1 - q2 * q2 + q3 * q3)
es[...,1] = np.arcsin((2 * (q0 * q1 + q2 * q3)).clip(-1,1))
es[...,2] = np.arctan2(2 * (q0 * q3 - q1 * q2), q0 * q0 - q1 * q1 + q2 * q2 - q3 * q3)
elif order == 'xzy':
es[...,0] = np.arctan2(2 * (q0 * q2 + q1 * q3), q0 * q0 + q1 * q1 - q2 * q2 - q3 * q3)
es[...,1] = np.arcsin((2 * (q0 * q3 - q1 * q2)).clip(-1,1))
es[...,2] = np.arctan2(2 * (q0 * q1 + q2 * q3), q0 * q0 - q1 * q1 + q2 * q2 - q3 * q3)
elif order == 'yxz':
es[...,0] = np.arctan2(2 * (q1 * q2 + q0 * q3), q0 * q0 - q1 * q1 + q2 * q2 - q3 * q3)
es[...,1] = np.arcsin((2 * (q0 * q1 - q2 * q3)).clip(-1,1))
es[...,2] = np.arctan2(2 * (q1 * q3 + q0 * q2), q0 * q0 - q1 * q1 - q2 * q2 + q3 * q3)
elif order == 'zyx':
es[...,0] = np.arctan2(2 * (q0 * q1 + q2 * q3), q0 * q0 - q1 * q1 - q2 * q2 + q3 * q3)
es[...,1] = np.arcsin((2 * (q0 * q2 - q1 * q3)).clip(-1,1))
es[...,2] = np.arctan2(2 * (q0 * q3 + q1 * q2), q0 * q0 + q1 * q1 - q2 * q2 - q3 * q3)
else:
raise KeyError('Unknown ordering %s' % order)
"""
# https://github.com/ehsan/ogre/blob/master/OgreMain/src/OgreMatrix3.cpp
# Use this class and convert from matrix
return es
def average(self):
if len(self.shape) == 1:
import numpy.core.umath_tests as ut
system = ut.matrix_multiply(self.qs[:,:,np.newaxis], self.qs[:,np.newaxis,:]).sum(axis=0)
w, v = np.linalg.eigh(system)
qiT_dot_qref = (self.qs[:,:,np.newaxis] * v[np.newaxis,:,:]).sum(axis=1)
return Quaternions(v[:,np.argmin((1.-qiT_dot_qref**2).sum(axis=0))])
else:
raise NotImplementedError('Cannot average multi-dimensionsal Quaternions')
def angle_axis(self):
norm = self.normalized()
s = np.sqrt(1 - (norm.reals**2.0))
s[s == 0] = 0.001
angles = 2.0 * np.arccos(norm.reals)
axis = norm.imaginaries / s[...,np.newaxis]
return angles, axis
def transforms(self):
qw = self.qs[...,0]
qx = self.qs[...,1]
qy = self.qs[...,2]
qz = self.qs[...,3]
x2 = qx + qx; y2 = qy + qy; z2 = qz + qz;
xx = qx * x2; yy = qy * y2; wx = qw * x2;
xy = qx * y2; yz = qy * z2; wy = qw * y2;
xz = qx * z2; zz = qz * z2; wz = qw * z2;
m = np.empty(self.shape + (3,3))
m[...,0,0] = 1.0 - (yy + zz)
m[...,0,1] = xy - wz
m[...,0,2] = xz + wy
m[...,1,0] = xy + wz
m[...,1,1] = 1.0 - (xx + zz)
m[...,1,2] = yz - wx
m[...,2,0] = xz - wy
m[...,2,1] = yz + wx
m[...,2,2] = 1.0 - (xx + yy)
return m
def ravel(self):
return self.qs.ravel()
@classmethod
def id(cls, n):
if isinstance(n, tuple):
qs = np.zeros(n + (4,))
qs[...,0] = 1.0
return Quaternions(qs)
if isinstance(n, int) or isinstance(n, long):
qs = np.zeros((n,4))
qs[:,0] = 1.0
return Quaternions(qs)
raise TypeError('Cannot Construct Quaternion from %s type' % str(type(n)))
@classmethod
def id_like(cls, a):
qs = np.zeros(a.shape + (4,))
qs[...,0] = 1.0
return Quaternions(qs)
@classmethod
def exp(cls, ws):
ts = np.sum(ws**2.0, axis=-1)**0.5
ts[ts == 0] = 0.001
ls = np.sin(ts) / ts
qs = np.empty(ws.shape[:-1] + (4,))
qs[...,0] = np.cos(ts)
qs[...,1] = ws[...,0] * ls
qs[...,2] = ws[...,1] * ls
qs[...,3] = ws[...,2] * ls
return Quaternions(qs).normalized()
@classmethod
def slerp(cls, q0s, q1s, a):
fst, snd = cls._broadcast(q0s.qs, q1s.qs)
fst, a = cls._broadcast(fst, a, scalar=True)
snd, a = cls._broadcast(snd, a, scalar=True)
len = np.sum(fst * snd, axis=-1)
neg = len < 0.0
len[neg] = -len[neg]
snd[neg] = -snd[neg]
amount0 = np.zeros(a.shape)
amount1 = np.zeros(a.shape)
linear = (1.0 - len) < 0.01
omegas = np.arccos(len[~linear])
sinoms = np.sin(omegas)
amount0[ linear] = 1.0 - a[linear]
amount1[ linear] = a[linear]
amount0[~linear] = np.sin((1.0 - a[~linear]) * omegas) / sinoms
amount1[~linear] = np.sin( a[~linear] * omegas) / sinoms
return Quaternions(
amount0[...,np.newaxis] * fst +
amount1[...,np.newaxis] * snd)
@classmethod
def between(cls, v0s, v1s):
a = np.cross(v0s, v1s)
w = np.sqrt((v0s**2).sum(axis=-1) * (v1s**2).sum(axis=-1)) + (v0s * v1s).sum(axis=-1)
return Quaternions(np.concatenate([w[...,np.newaxis], a], axis=-1)).normalized()
@classmethod
def from_angle_axis(cls, angles, axis):
axis = axis / (np.sqrt(np.sum(axis**2, axis=-1)) + 1e-10)[...,np.newaxis]
sines = np.sin(angles / 2.0)[...,np.newaxis]
cosines = np.cos(angles / 2.0)[...,np.newaxis]
return Quaternions(np.concatenate([cosines, axis * sines], axis=-1))
@classmethod
def from_euler(cls, es, order='xyz', world=False):
axis = {
'x' : np.array([1,0,0]),
'y' : np.array([0,1,0]),
'z' : np.array([0,0,1]),
}
q0s = Quaternions.from_angle_axis(es[...,0], axis[order[0]])
q1s = Quaternions.from_angle_axis(es[...,1], axis[order[1]])
q2s = Quaternions.from_angle_axis(es[...,2], axis[order[2]])
return (q2s * (q1s * q0s)) if world else (q0s * (q1s * q2s))
@classmethod
def from_transforms(cls, ts):
d0, d1, d2 = ts[...,0,0], ts[...,1,1], ts[...,2,2]
q0 = ( d0 + d1 + d2 + 1.0) / 4.0
q1 = ( d0 - d1 - d2 + 1.0) / 4.0
q2 = (-d0 + d1 - d2 + 1.0) / 4.0
q3 = (-d0 - d1 + d2 + 1.0) / 4.0
q0 = np.sqrt(q0.clip(0,None))
q1 = np.sqrt(q1.clip(0,None))
q2 = np.sqrt(q2.clip(0,None))
q3 = np.sqrt(q3.clip(0,None))
c0 = (q0 >= q1) & (q0 >= q2) & (q0 >= q3)
c1 = (q1 >= q0) & (q1 >= q2) & (q1 >= q3)
c2 = (q2 >= q0) & (q2 >= q1) & (q2 >= q3)
c3 = (q3 >= q0) & (q3 >= q1) & (q3 >= q2)
q1[c0] *= np.sign(ts[c0,2,1] - ts[c0,1,2])
q2[c0] *= np.sign(ts[c0,0,2] - ts[c0,2,0])
q3[c0] *= np.sign(ts[c0,1,0] - ts[c0,0,1])
q0[c1] *= np.sign(ts[c1,2,1] - ts[c1,1,2])
q2[c1] *= np.sign(ts[c1,1,0] + ts[c1,0,1])
q3[c1] *= np.sign(ts[c1,0,2] + ts[c1,2,0])
q0[c2] *= np.sign(ts[c2,0,2] - ts[c2,2,0])
q1[c2] *= np.sign(ts[c2,1,0] + ts[c2,0,1])
q3[c2] *= np.sign(ts[c2,2,1] + ts[c2,1,2])
q0[c3] *= np.sign(ts[c3,1,0] - ts[c3,0,1])
q1[c3] *= np.sign(ts[c3,2,0] + ts[c3,0,2])
q2[c3] *= np.sign(ts[c3,2,1] + ts[c3,1,2])
qs = np.empty(ts.shape[:-2] + (4,))
qs[...,0] = q0
qs[...,1] = q1
qs[...,2] = q2
qs[...,3] = q3
return cls(qs)
================================================
FILE: pymo/__init__.py
================================================
================================================
FILE: pymo/data.py
================================================
import numpy as np
class Joint():
def __init__(self, name, parent=None, children=None):
self.name = name
self.parent = parent
self.children = children
class MocapData():
def __init__(self):
self.skeleton = {}
self.values = None
self.channel_names = []
self.framerate = 0.0
self.root_name = ''
self.take_name = ''
def traverse(self, j=None):
stack = [self.root_name]
while stack:
joint = stack.pop()
yield joint
for c in self.skeleton[joint]['children']:
stack.append(c)
def clone(self):
import copy
new_data = MocapData()
new_data.skeleton = copy.deepcopy(self.skeleton)
new_data.values = copy.deepcopy(self.values)
new_data.channel_names = copy.deepcopy(self.channel_names)
new_data.root_name = copy.deepcopy(self.root_name)
new_data.framerate = copy.deepcopy(self.framerate)
if hasattr(self,'take_name'):
new_data.take_name = copy.deepcopy(self.take_name)
return new_data
def get_all_channels(self):
'''Returns all of the channels parsed from the file as a 2D numpy array'''
frames = [f[1] for f in self.values]
return np.asarray([[channel[2] for channel in frame] for frame in frames])
def get_skeleton_tree(self):
tree = []
root_key = [j for j in self.skeleton if self.skeleton[j]['parent']==None][0]
root_joint = Joint(root_key)
def get_empty_channels(self):
#TODO
pass
def get_constant_channels(self):
#TODO
pass
================================================
FILE: pymo/features.py
================================================
'''
A set of mocap feature extraction functions
Created by Omid Alemi | Nov 17 2017
'''
import numpy as np
import pandas as pd
import peakutils
import matplotlib.pyplot as plt
def get_foot_contact_idxs(signal, t=0.02, min_dist=120):
up_idxs = peakutils.indexes(signal, thres=t/max(signal), min_dist=min_dist)
down_idxs = peakutils.indexes(-signal, thres=t/min(signal), min_dist=min_dist)
return [up_idxs, down_idxs]
def create_foot_contact_signal(mocap_track, col_name, start=1, t=0.02, min_dist=120):
signal = mocap_track.values[col_name].values
idxs = get_foot_contact_idxs(signal, t, min_dist)
step_signal = []
c = start
for f in range(len(signal)):
if f in idxs[1]:
c = 0
elif f in idxs[0]:
c = 1
step_signal.append(c)
return step_signal
def plot_foot_up_down(mocap_track, col_name, t=0.02, min_dist=120):
signal = mocap_track.values[col_name].values
idxs = get_foot_contact_idxs(signal, t, min_dist)
plt.plot(mocap_track.values.index, signal)
plt.plot(mocap_track.values.index[idxs[0]], signal[idxs[0]], 'ro')
plt.plot(mocap_track.values.index[idxs[1]], signal[idxs[1]], 'go')
================================================
FILE: pymo/parsers.py
================================================
'''
BVH Parser Class
By Omid Alemi
Created: June 12, 2017
Based on: https://gist.github.com/johnfredcee/2007503
'''
import os
import re
import numpy as np
from pymo.data import Joint, MocapData
class BVHScanner():
'''
A wrapper class for re.Scanner
'''
def __init__(self):
def identifier(scanner, token):
return 'IDENT', token
def operator(scanner, token):
return 'OPERATOR', token
def digit(scanner, token):
return 'DIGIT', token
def open_brace(scanner, token):
return 'OPEN_BRACE', token
def close_brace(scanner, token):
return 'CLOSE_BRACE', token
self.scanner = re.Scanner([
(r'[a-zA-Z_]\w*', identifier),
#(r'-*[0-9]+(\.[0-9]+)?', digit), # won't work for .34
#(r'[-+]?[0-9]*\.?[0-9]+', digit), # won't work for 4.56e-2
#(r'[-+]?[0-9]*\.?[0-9]+([eE][-+]?[0-9]+)?', digit),
(r'-*[0-9]*\.?[0-9]+([eE][-+]?[0-9]+)?', digit),
(r'}', close_brace),
(r'}', close_brace),
(r'{', open_brace),
(r':', None),
(r'\s+', None)
])
def scan(self, stuff):
return self.scanner.scan(stuff)
class BVHParser():
'''
A class to parse a BVH file.
Extracts the skeleton and channel values
'''
def __init__(self, filename=None):
self.reset()
def reset(self):
self._skeleton = {}
self.bone_context = []
self._motion_channels = []
self._motions = []
self.current_token = 0
self.framerate = 0.0
self.root_name = ''
self.scanner = BVHScanner()
self.data = MocapData()
def parse(self, filename, start=0, stop=-1):
self.reset()
with open(filename, 'r') as bvh_file:
raw_contents = bvh_file.read()
tokens, remainder = self.scanner.scan(raw_contents)
self._parse_hierarchy(tokens)
self.current_token = self.current_token + 1
self._parse_motion(tokens, start, stop)
self.data.skeleton = self._skeleton
self.data.channel_names = self._motion_channels
self.data.values = self._to_DataFrame()
self.data.root_name = self.root_name
self.data.framerate = self.framerate
self.data.take_name = os.path.basename(os.path.splitext(filename)[0])
return self.data
def _to_DataFrame(self):
'''Returns all of the channels parsed from the file as a pandas DataFrame'''
import pandas as pd
time_index = pd.to_timedelta([f[0] for f in self._motions], unit='s')
frames = [f[1] for f in self._motions]
channels = np.asarray([[channel[2] for channel in frame] for frame in frames])
column_names = ['%s_%s'%(c[0], c[1]) for c in self._motion_channels]
return pd.DataFrame(data=channels, index=time_index, columns=column_names)
def _new_bone(self, parent, name):
bone = {'parent': parent, 'channels': [], 'offsets': [], 'order': '','children': []}
return bone
def _push_bone_context(self,name):
self.bone_context.append(name)
def _get_bone_context(self):
return self.bone_context[len(self.bone_context)-1]
def _pop_bone_context(self):
self.bone_context = self.bone_context[:-1]
return self.bone_context[len(self.bone_context)-1]
def _read_offset(self, bvh, token_index):
if bvh[token_index] != ('IDENT', 'OFFSET'):
return None, None
token_index = token_index + 1
offsets = [0.0] * 3
for i in range(3):
offsets[i] = float(bvh[token_index][1])
token_index = token_index + 1
return offsets, token_index
def _read_channels(self, bvh, token_index):
if bvh[token_index] != ('IDENT', 'CHANNELS'):
return None, None
token_index = token_index + 1
channel_count = int(bvh[token_index][1])
token_index = token_index + 1
channels = [""] * channel_count
order = ""
for i in range(channel_count):
channels[i] = bvh[token_index][1]
token_index = token_index + 1
if(channels[i] == "Xrotation" or channels[i]== "Yrotation" or channels[i]== "Zrotation"):
order += channels[i][0]
else :
order = ""
return channels, token_index, order
def _parse_joint(self, bvh, token_index):
end_site = False
joint_id = bvh[token_index][1]
token_index = token_index + 1
joint_name = bvh[token_index][1]
token_index = token_index + 1
parent_name = self._get_bone_context()
if (joint_id == "End"):
joint_name = parent_name+ '_Nub'
end_site = True
joint = self._new_bone(parent_name, joint_name)
if bvh[token_index][0] != 'OPEN_BRACE':
print('Was expecting brance, got ', bvh[token_index])
return None
token_index = token_index + 1
offsets, token_index = self._read_offset(bvh, token_index)
joint['offsets'] = offsets
if not end_site:
channels, token_index, order = self._read_channels(bvh, token_index)
joint['channels'] = channels
joint['order'] = order
for channel in channels:
self._motion_channels.append((joint_name, channel))
self._skeleton[joint_name] = joint
self._skeleton[parent_name]['children'].append(joint_name)
while (bvh[token_index][0] == 'IDENT' and bvh[token_index][1] == 'JOINT') or (bvh[token_index][0] == 'IDENT' and bvh[token_index][1] == 'End'):
self._push_bone_context(joint_name)
token_index = self._parse_joint(bvh, token_index)
self._pop_bone_context()
if bvh[token_index][0] == 'CLOSE_BRACE':
return token_index + 1
print('Unexpected token ', bvh[token_index])
def _parse_hierarchy(self, bvh):
self.current_token = 0
if bvh[self.current_token] != ('IDENT', 'HIERARCHY'):
return None
self.current_token = self.current_token + 1
if bvh[self.current_token] != ('IDENT', 'ROOT'):
return None
self.current_token = self.current_token + 1
if bvh[self.current_token][0] != 'IDENT':
return None
root_name = bvh[self.current_token][1]
root_bone = self._new_bone(None, root_name)
self.current_token = self.current_token + 2 #skipping open brace
offsets, self.current_token = self._read_offset(bvh, self.current_token)
channels, self.current_token, order = self._read_channels(bvh, self.current_token)
root_bone['offsets'] = offsets
root_bone['channels'] = channels
root_bone['order'] = order
self._skeleton[root_name] = root_bone
self._push_bone_context(root_name)
for channel in channels:
self._motion_channels.append((root_name, channel))
while bvh[self.current_token][1] == 'JOINT' or bvh[self.current_token][1] == 'End':
self.current_token = self._parse_joint(bvh, self.current_token)
self.root_name = root_name
def _parse_motion(self, bvh, start, stop):
if bvh[self.current_token][0] != 'IDENT':
print('Unexpected text')
return None
if bvh[self.current_token][1] != 'MOTION':
print('No motion section')
return None
self.current_token = self.current_token + 1
if bvh[self.current_token][1] != 'Frames':
return None
self.current_token = self.current_token + 1
frame_count = int(bvh[self.current_token][1])
if stop<0 or stop>frame_count:
stop = frame_count
assert(start>=0)
assert(start<stop)
self.current_token = self.current_token + 1
if bvh[self.current_token][1] != 'Frame':
return None
self.current_token = self.current_token + 1
if bvh[self.current_token][1] != 'Time':
return None
self.current_token = self.current_token + 1
frame_rate = float(bvh[self.current_token][1])
self.framerate = frame_rate
self.current_token = self.current_token + 1
frame_time = 0.0
self._motions = [()] * (stop-start)
idx=0
for i in range(stop):
channel_values = []
for channel in self._motion_channels:
channel_values.append((channel[0], channel[1], float(bvh[self.current_token][1])))
self.current_token = self.current_token + 1
if i>=start:
self._motions[idx] = (frame_time, channel_values)
frame_time = frame_time + frame_rate
idx+=1
================================================
FILE: pymo/preprocessing.py
================================================
'''
Preprocessing Tranformers Based on sci-kit's API
By Omid Alemi
Created on June 12, 2017
'''
import copy
import pandas as pd
import numpy as np
import transforms3d as t3d
import scipy.ndimage.filters as filters
from scipy.spatial.transform import Rotation as R
from scipy import signal, interpolate
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.pipeline import Pipeline
from pymo.rotation_tools import Rotation, euler2expmap, euler2expmap2, expmap2euler, euler_reorder, unroll, euler2vectors, vectors2euler
from pymo.Quaternions import Quaternions
from pymo.Pivots import Pivots
class MocapParameterizer(BaseEstimator, TransformerMixin):
def __init__(self, param_type = 'euler', ref_pose=None):
'''
param_type = {'euler', 'quat', 'expmap', 'position', 'expmap2pos'}
'''
self.param_type = param_type
if (ref_pose is not None):
self.ref_pose = self._to_quat(ref_pose)[0]
else:
self.ref_pose = None
def fit(self, X, y=None):
return self
def transform(self, X, y=None):
#print("MocapParameterizer: " + self.param_type)
if self.param_type == 'euler':
return X
elif self.param_type == 'expmap':
if self.ref_pose is None:
return self._to_expmap(X)
else:
return self._to_expmap2(X)
elif self.param_type == 'vectors':
return self._euler_to_vectors(X)
elif self.param_type == 'quat':
return self._to_quat(X)
elif self.param_type == 'position':
return self._to_pos(X)
elif self.param_type == 'expmap2pos':
return self._expmap_to_pos(X)
else:
raise 'param types: euler, quat, expmap, position, expmap2pos'
# return X
def inverse_transform(self, X, copy=None):
if self.param_type == 'euler':
return X
elif self.param_type == 'expmap':
if self.ref_pose is None:
return self._expmap_to_euler(X)
else:
return self._expmap_to_euler2(X)
elif self.param_type == 'vectors':
return self._vectors_to_euler(X)
elif self.param_type == 'quat':
return self._quat_to_euler(X)
elif self.param_type == 'position':
# raise 'positions 2 eulers is not supported'
print('positions 2 eulers is not supported')
return X
else:
raise 'param types: euler, quat, expmap, position'
def _to_quat(self, X):
'''Converts joints rotations in quaternions'''
Q = []
for track in X:
channels = []
titles = []
euler_df = track.values
# Create a new DataFrame to store the exponential map rep
quat_df = euler_df.copy()
# List the columns that contain rotation channels
rot_cols = [c for c in euler_df.columns if ('rotation' in c and 'Nub' not in c)]
# List the joints that are not end sites, i.e., have channels
joints = (joint for joint in track.skeleton if 'Nub' not in joint)
for joint in joints:
rot_order = track.skeleton[joint]['order']
# Get the rotation columns that belong to this joint
rc = euler_df[[c for c in rot_cols if joint in c]]
r1_col = '%s_%srotation'%(joint, rot_order[0])
r2_col = '%s_%srotation'%(joint, rot_order[1])
r3_col = '%s_%srotation'%(joint, rot_order[2])
# Make sure the columns are organized in xyz order
if rc.shape[1] < 3:
euler_values = np.zeros((euler_df.shape[0], 3))
rot_order = "XYZ"
else:
euler_values = np.pi/180.0*np.transpose(np.array([track.values[r1_col], track.values[r2_col], track.values[r3_col]]))
quat_df.drop([r1_col, r2_col, r3_col], axis=1, inplace=True)
quats = Quaternions.from_euler(np.asarray(euler_values), order=rot_order.lower(), world=False)
# Create the corresponding columns in the new DataFrame
quat_df['%s_qWrotation'%joint] = pd.Series(data=[e[0] for e in quats], index=quat_df.index)
quat_df['%s_qXrotation'%joint] = pd.Series(data=[e[1] for e in quats], index=quat_df.index)
quat_df['%s_qYrotation'%joint] = pd.Series(data=[e[2] for e in quats], index=quat_df.index)
quat_df['%s_qZrotation'%joint] = pd.Series(data=[e[3] for e in quats], index=quat_df.index)
new_track = track.clone()
new_track.values = quat_df
Q.append(new_track)
return Q
def _quat_to_euler(self, X):
Q = []
for track in X:
channels = []
titles = []
quat_df = track.values
# Create a new DataFrame to store the exponential map rep
#euler_df = pd.DataFrame(index=exp_df.index)
euler_df = quat_df.copy()
# List the columns that contain rotation channels
quat_params = [c for c in quat_df.columns if ( any(p in c for p in ['qWrotation','qXrotation','qYrotation','qZrotation']) and 'Nub' not in c)]
# List the joints that are not end sites, i.e., have channels
joints = (joint for joint in track.skeleton if 'Nub' not in joint)
for joint in joints:
r = quat_df[[c for c in quat_params if joint in c]] # Get the columns that belong to this joint
euler_df.drop(['%s_qWrotation'%joint, '%s_qXrotation'%joint, '%s_qYrotation'%joint, '%s_qZrotation'%joint], axis=1, inplace=True)
quat = [[f[1]['%s_qWrotation'%joint], f[1]['%s_qXrotation'%joint], f[1]['%s_qYrotation'%joint], f[1]['%s_qZrotation'%joint]] for f in r.iterrows()] # Make sure the columsn are organized in xyz order
quats=Quaternions(np.asarray(quat))
euler_rots = 180/np.pi*quats.euler()
track.skeleton[joint]['order'] = 'ZYX'
rot_order = track.skeleton[joint]['order']
#euler_rots = [Rotation(f, 'expmap').to_euler(True, rot_order) for f in expmap] # Convert the exp maps to eulers
#euler_rots = [expmap2euler(f, rot_order, True) for f in expmap] # Convert the exp maps to eulers
# Create the corresponding columns in the new DataFrame
euler_df['%s_%srotation'%(joint, rot_order[2])] = pd.Series(data=[e[0] for e in euler_rots], index=euler_df.index)
euler_df['%s_%srotation'%(joint, rot_order[1])] = pd.Series(data=[e[1] for e in euler_rots], index=euler_df.index)
euler_df['%s_%srotation'%(joint, rot_order[0])] = pd.Series(data=[e[2] for e in euler_rots], index=euler_df.index)
new_track = track.clone()
new_track.values = euler_df
Q.append(new_track)
return Q
def _to_pos(self, X):
'''Converts joints rotations in Euler angles to joint positions'''
Q = []
for track in X:
channels = []
titles = []
euler_df = track.values
# Create a new DataFrame to store the exponential map rep
pos_df = pd.DataFrame(index=euler_df.index)
# List the columns that contain rotation channels
rot_cols = [c for c in euler_df.columns if ('rotation' in c)]
# List the columns that contain position channels
pos_cols = [c for c in euler_df.columns if ('position' in c)]
# List the joints that are not end sites, i.e., have channels
joints = (joint for joint in track.skeleton)
tree_data = {}
for joint in track.traverse():
parent = track.skeleton[joint]['parent']
rot_order = track.skeleton[joint]['order']
#print("rot_order:" + joint + " :" + rot_order)
# Get the rotation columns that belong to this joint
rc = euler_df[[c for c in rot_cols if joint in c]]
# Get the position columns that belong to this joint
pc = euler_df[[c for c in pos_cols if joint in c]]
# Make sure the columns are organized in xyz order
if rc.shape[1] < 3:
euler_values = np.zeros((euler_df.shape[0], 3))
rot_order = "XYZ"
else:
euler_values = np.pi/180.0*np.transpose(np.array([track.values['%s_%srotation'%(joint, rot_order[0])], track.values['%s_%srotation'%(joint, rot_order[1])], track.values['%s_%srotation'%(joint, rot_order[2])]]))
if pc.shape[1] < 3:
pos_values = np.asarray([[0,0,0] for f in pc.iterrows()])
else:
pos_values =np.asarray([[f[1]['%s_Xposition'%joint],
f[1]['%s_Yposition'%joint],
f[1]['%s_Zposition'%joint]] for f in pc.iterrows()])
quats = Quaternions.from_euler(np.asarray(euler_values), order=rot_order.lower(), world=False)
tree_data[joint]=[
[], # to store the rotation matrix
[] # to store the calculated position
]
if track.root_name == joint:
tree_data[joint][0] = quats#rotmats
# tree_data[joint][1] = np.add(pos_values, track.skeleton[joint]['offsets'])
tree_data[joint][1] = pos_values
else:
# for every frame i, multiply this joint's rotmat to the rotmat of its parent
tree_data[joint][0] = tree_data[parent][0]*quats# np.matmul(rotmats, tree_data[parent][0])
# add the position channel to the offset and store it in k, for every frame i
k = pos_values + np.asarray(track.skeleton[joint]['offsets'])
# multiply k to the rotmat of the parent for every frame i
q = tree_data[parent][0]*k #np.matmul(k.reshape(k.shape[0],1,3), tree_data[parent][0])
# add q to the position of the parent, for every frame i
tree_data[joint][1] = tree_data[parent][1] + q #q.reshape(k.shape[0],3) + tree_data[parent][1]
# Create the corresponding columns in the new DataFrame
df = pd.DataFrame(data=tree_data[joint][1],
index=pos_df.index,
columns=['%s_Xposition'%joint, '%s_Yposition'%joint, '%s_Zposition'%joint])
pos_df = pd.concat((pos_df, df), axis=1)
new_track = track.clone()
new_track.values = pos_df
Q.append(new_track)
return Q
def _expmap2rot(self, expmap):
theta = np.linalg.norm(expmap, axis=1, keepdims=True)
nz = np.nonzero(theta)[0]
expmap[nz,:] = expmap[nz,:]/theta[nz]
nrows=expmap.shape[0]
x = expmap[:,0]
y = expmap[:,1]
z = expmap[:,2]
s = np.sin(theta*0.5).reshape(nrows)
c = np.cos(theta*0.5).reshape(nrows)
rotmats = np.zeros((nrows, 3, 3))
rotmats[:,0,0] = 2*(x*x-1)*s*s+1
rotmats[:,0,1] = 2*x*y*s*s-2*z*c*s
rotmats[:,0,2] = 2*x*z*s*s+2*y*c*s
rotmats[:,1,0] = 2*x*y*s*s+2*z*c*s
rotmats[:,1,1] = 2*(y*y-1)*s*s+1
rotmats[:,1,2] = 2*y*z*s*s-2*x*c*s
rotmats[:,2,0] = 2*x*z*s*s-2*y*c*s
rotmats[:,2,1] = 2*y*z*s*s+2*x*c*s
rotmats[:,2,2] = 2*(z*z-1)*s*s+1
return rotmats
def _expmap_to_pos(self, X):
'''Converts joints rotations in expmap notation to joint positions'''
Q = []
for track in X:
channels = []
titles = []
exp_df = track.values
# Create a new DataFrame to store the exponential map rep
pos_df = pd.DataFrame(index=exp_df.index)
# List the columns that contain rotation channels
exp_params = [c for c in exp_df.columns if ( any(p in c for p in ['alpha', 'beta','gamma']) and 'Nub' not in c)]
# List the joints that are not end sites, i.e., have channels
joints = (joint for joint in track.skeleton)
tree_data = {}
for joint in track.traverse():
parent = track.skeleton[joint]['parent']
if 'Nub' not in joint:
r = exp_df[[c for c in exp_params if joint in c]] # Get the columns that belong to this joint
expmap = r.values
#expmap = [[f[1]['%s_alpha'%joint], f[1]['%s_beta'%joint], f[1]['%s_gamma'%joint]] for f in r.iterrows()]
else:
expmap = np.zeros((exp_df.shape[0], 3))
# Convert the eulers to rotation matrices
#rotmats = np.asarray([Rotation(f, 'expmap').rotmat for f in expmap])
#angs = np.linalg.norm(expmap,axis=1, keepdims=True)
rotmats = self._expmap2rot(expmap)
tree_data[joint]=[
[], # to store the rotation matrix
[] # to store the calculated position
]
pos_values = np.zeros((exp_df.shape[0], 3))
if track.root_name == joint:
tree_data[joint][0] = rotmats
# tree_data[joint][1] = np.add(pos_values, track.skeleton[joint]['offsets'])
tree_data[joint][1] = pos_values
else:
# for every frame i, multiply this joint's rotmat to the rotmat of its parent
tree_data[joint][0] = np.matmul(rotmats, tree_data[parent][0])
# add the position channel to the offset and store it in k, for every frame i
k = pos_values + track.skeleton[joint]['offsets']
# multiply k to the rotmat of the parent for every frame i
q = np.matmul(k.reshape(k.shape[0],1,3), tree_data[parent][0])
# add q to the position of the parent, for every frame i
tree_data[joint][1] = q.reshape(k.shape[0],3) + tree_data[parent][1]
# Create the corresponding columns in the new DataFrame
df = pd.DataFrame(data=tree_data[joint][1],
index=pos_df.index,
columns=['%s_Xposition'%joint, '%s_Yposition'%joint, '%s_Zposition'%joint])
pos_df = pd.concat((pos_df, df), axis=1)
new_track = track.clone()
new_track.values = pos_df
Q.append(new_track)
return Q
def _to_expmap(self, X):
'''Converts Euler angles to Exponential Maps'''
Q = []
for track in X:
channels = []
titles = []
euler_df = track.values
# Create a new DataFrame to store the exponential map rep
exp_df = euler_df.copy()# pd.DataFrame(index=euler_df.index)
# List the columns that contain rotation channels
rots = [c for c in euler_df.columns if ('rotation' in c and 'Nub' not in c)]
# List the joints that are not end sites, i.e., have channels
joints = (joint for joint in track.skeleton if 'Nub' not in joint)
for joint in joints:
#print(joint)
r = euler_df[[c for c in rots if joint in c]] # Get the columns that belong to this joint
rot_order = track.skeleton[joint]['order']
r1_col = '%s_%srotation'%(joint, rot_order[0])
r2_col = '%s_%srotation'%(joint, rot_order[1])
r3_col = '%s_%srotation'%(joint, rot_order[2])
exp_df.drop([r1_col, r2_col, r3_col], axis=1, inplace=True)
euler = np.transpose(np.array([r[r1_col], r[r2_col], r[r3_col]]))
#exps = [Rotation(f, 'euler', from_deg=True, order=rot_order).to_expmap() for f in euler] # Convert the eulers to exp maps
exps = unroll(np.array([euler2expmap(f, rot_order, True) for f in euler])) # Convert the exp maps to eulers
#exps = euler2expmap2(euler, rot_order, True) # Convert the eulers to exp maps
# Create the corresponding columns in the new DataFrame
exp_df.insert(loc=0, column='%s_gamma'%joint, value=pd.Series(data=[e[2] for e in exps], index=exp_df.index))
exp_df.insert(loc=0, column='%s_beta'%joint, value=pd.Series(data=[e[1] for e in exps], index=exp_df.index))
exp_df.insert(loc=0, column='%s_alpha'%joint, value=pd.Series(data=[e[0] for e in exps], index=exp_df.index))
#print(exp_df.columns)
new_track = track.clone()
new_track.values = exp_df
Q.append(new_track)
return Q
def _expmap_to_euler(self, X):
Q = []
for track in X:
channels = []
titles = []
exp_df = track.values
# Create a new DataFrame to store the exponential map rep
euler_df = exp_df.copy()
# List the columns that contain rotation channels
exp_params = [c for c in exp_df.columns if ( any(p in c for p in ['alpha', 'beta','gamma']) and 'Nub' not in c)]
# List the joints that are not end sites, i.e., have channels
joints = (joint for joint in track.skeleton if 'Nub' not in joint)
for joint in joints:
r = exp_df[[c for c in exp_params if joint in c]] # Get the columns that belong to this joint
jt_alpha = '%s_alpha'%joint
jt_beta = '%s_beta'%joint
jt_gamma = '%s_gamma'%joint
euler_df.drop([jt_alpha, jt_beta, jt_gamma], axis=1, inplace=True)
expmap = np.transpose(np.array([track.values[jt_alpha], track.values[jt_beta], track.values[jt_gamma]]))
rot_order = track.skeleton[joint]['order']
euler_rots = np.array(R.from_rotvec(expmap).as_euler(rot_order, degrees=True))
#euler_rots = [expmap2euler(f, rot_order, True) for f in expmap] # Convert the exp maps to eulers
# Create the corresponding columns in the new DataFrame
euler_df['%s_%srotation'%(joint, rot_order[0])] = pd.Series(data=[e[0] for e in euler_rots], index=euler_df.index)
euler_df['%s_%srotation'%(joint, rot_order[1])] = pd.Series(data=[e[1] for e in euler_rots], index=euler_df.index)
euler_df['%s_%srotation'%(joint, rot_order[2])] = pd.Series(data=[e[2] for e in euler_rots], index=euler_df.index)
new_track = track.clone()
new_track.values = euler_df
Q.append(new_track)
return Q
def _to_expmap2(self, X):
'''Converts Euler angles to Exponential Maps'''
Q = []
for track in X:
channels = []
titles = []
euler_df = track.values
# Create a new DataFrame to store the exponential map rep
exp_df = euler_df.copy()# pd.DataFrame(index=euler_df.index)
# Copy the root positions into the new DataFrame
#rxp = '%s_Xposition'%track.root_name
#ryp = '%s_Yposition'%track.root_name
#rzp = '%s_Zposition'%track.root_name
#exp_df[rxp] = pd.Series(data=euler_df[rxp], index=exp_df.index)
#exp_df[ryp] = pd.Series(data=euler_df[ryp], index=exp_df.index)
#exp_df[rzp] = pd.Series(data=euler_df[rzp], index=exp_df.index)
# List the columns that contain rotation channels
rots = [c for c in euler_df.columns if ('rotation' in c and 'Nub' not in c)]
# List the joints that are not end sites, i.e., have channels
joints = (joint for joint in track.skeleton if 'Nub' not in joint)
for joint in joints:
r = euler_df[[c for c in rots if joint in c]] # Get the columns that belong to this joint
rot_order = track.skeleton[joint]['order']
# Get the rotation columns that belong to this joint
rc = euler_df[[c for c in rots if joint in c]]
r1_col = '%s_%srotation'%(joint, rot_order[0])
r2_col = '%s_%srotation'%(joint, rot_order[1])
r3_col = '%s_%srotation'%(joint, rot_order[2])
# Make sure the columns are organized in xyz order
#print("joint:" + str(joint) + " rot_order:" + str(rot_order))
if rc.shape[1] < 3:
euler_values = np.zeros((euler_df.shape[0], 3))
rot_order = "XYZ"
else:
euler_values = np.pi/180.0*np.transpose(np.array([track.values[r1_col], track.values[r2_col], track.values[r3_col]]))
quats = Quaternions.from_euler(np.asarray(euler_values), order=rot_order.lower(), world=False)
#exps = [Rotation(f, 'euler', from_deg=True, order=rot_order).to_expmap() for f in euler] # Convert the eulers to exp maps
#exps = unroll(np.array([euler2expmap(f, rot_order, True) for f in euler])) # Convert the exp maps to eulers
#exps = euler2expmap2(euler, rot_order, True) # Convert the eulers to exp maps
# Create the corresponding columns in the new DataFrame
if (self.ref_pose is not None):
q1_col = '%s_qWrotation'%(joint)
q2_col = '%s_qXrotation'%(joint)
q3_col = '%s_qYrotation'%(joint)
q4_col = '%s_qZrotation'%(joint)
ref_q = Quaternions(np.asarray([[f[1][q1_col], f[1][q2_col], f[1][q3_col], f[1][q4_col]] for f in self.ref_pose.values.iterrows()]))
#print("ref_q:" + str(ref_q.shape))
ref_q = ref_q[0,:]
quats=(-ref_q)*quats
angles, axis = quats.angle_axis()
aa = np.where(angles>np.pi)
angles[aa] = angles[aa]-2*np.pi
#exps = unroll(angles[:,None]*axis)
exps = angles[:,None]*axis
#print(f"{joint}: {str(exps[0,:])}")
#exps = np.array([quat2expmap(f) for f in quats])
exp_df.drop([r1_col, r2_col, r3_col], axis=1, inplace=True)
exp_df.insert(loc=0, column='%s_gamma'%joint, value=pd.Series(data=[e[2] for e in exps], index=exp_df.index))
exp_df.insert(loc=0, column='%s_beta'%joint, value=pd.Series(data=[e[1] for e in exps], index=exp_df.index))
exp_df.insert(loc=0, column='%s_alpha'%joint, value=pd.Series(data=[e[0] for e in exps], index=exp_df.index))
#print(exp_df.columns)
new_track = track.clone()
new_track.values = exp_df
Q.append(new_track)
return Q
def _expmap_to_euler2(self, X):
Q = []
for track in X:
channels = []
titles = []
exp_df = track.values
# Create a new DataFrame to store the exponential map rep
#euler_df = pd.DataFrame(index=exp_df.index)
euler_df = exp_df.copy()
# Copy the root positions into the new DataFrame
#rxp = '%s_Xposition'%track.root_name
#ryp = '%s_Yposition'%track.root_name
#rzp = '%s_Zposition'%track.root_name
#euler_df[rxp] = pd.Series(data=exp_df[rxp], index=euler_df.index)
#euler_df[ryp] = pd.Series(data=exp_df[ryp], index=euler_df.index)
#euler_df[rzp] = pd.Series(data=exp_df[rzp], index=euler_df.index)
# List the columns that contain rotation channels
exp_params = [c for c in exp_df.columns if ( any(p in c for p in ['alpha', 'beta','gamma']) and 'Nub' not in c)]
# List the joints that are not end sites, i.e., have channels
joints = (joint for joint in track.skeleton if 'Nub' not in joint)
for joint in joints:
r = exp_df[[c for c in exp_params if joint in c]] # Get the columns that belong to this joint
euler_df.drop(['%s_alpha'%joint, '%s_beta'%joint, '%s_gamma'%joint], axis=1, inplace=True)
expmap = [[f[1]['%s_alpha'%joint], f[1]['%s_beta'%joint], f[1]['%s_gamma'%joint]] for f in r.iterrows()] # Make sure the columsn are organized in xyz order
angs = np.linalg.norm(expmap, axis=1)
quats=Quaternions.from_angle_axis(angs, expmap/(np.tile(angs[:, None]+1e-10, (1,3))))
if (self.ref_pose is not None):
q1_col = '%s_qWrotation'%(joint)
q2_col = '%s_qXrotation'%(joint)
q3_col = '%s_qYrotation'%(joint)
q4_col = '%s_qZrotation'%(joint)
ref_q = Quaternions(np.asarray([[f[1][q1_col], f[1][q2_col], f[1][q3_col], f[1][q4_col]] for f in self.ref_pose.values.iterrows()]))
#print("ref_q:" + str(ref_q.shape))
ref_q = ref_q[0,:]
quats=ref_q*quats
euler_rots = 180/np.pi*quats.euler()
track.skeleton[joint]['order'] = 'ZYX'
rot_order = track.skeleton[joint]['order']
#euler_rots = [Rotation(f, 'expmap').to_euler(True, rot_order) for f in expmap] # Convert the exp maps to eulers
#euler_rots = [expmap2euler(f, rot_order, True) for f in expmap] # Convert the exp maps to eulers
# Create the corresponding columns in the new DataFrame
euler_df['%s_%srotation'%(joint, rot_order[2])] = pd.Series(data=[e[0] for e in euler_rots], index=euler_df.index)
euler_df['%s_%srotation'%(joint, rot_order[1])] = pd.Series(data=[e[1] for e in euler_rots], index=euler_df.index)
euler_df['%s_%srotation'%(joint, rot_order[0])] = pd.Series(data=[e[2] for e in euler_rots], index=euler_df.index)
new_track = track.clone()
new_track.values = euler_df
Q.append(new_track)
return Q
def _euler_to_vectors(self, X):
'''Converts Euler angles to Up and Fwd vectors'''
Q = []
for track in X:
channels = []
titles = []
euler_df = track.values
# Create a new DataFrame to store the exponential map rep
vec_df = euler_df.copy()# pd.DataFrame(index=euler_df.index)
# List the columns that contain rotation channels
rots = [c for c in euler_df.columns if ('rotation' in c and 'Nub' not in c)]
# List the joints that are not end sites, i.e., have channels
joints = (joint for joint in track.skeleton if 'Nub' not in joint)
for joint in joints:
#print(joint)
r = euler_df[[c for c in rots if joint in c]] # Get the columns that belong to this joint
rot_order = track.skeleton[joint]['order']
r1_col = '%s_%srotation'%(joint, rot_order[0])
r2_col = '%s_%srotation'%(joint, rot_order[1])
r3_col = '%s_%srotation'%(joint, rot_order[2])
vec_df.drop([r1_col, r2_col, r3_col], axis=1, inplace=True)
euler = [[f[1][r1_col], f[1][r2_col], f[1][r3_col]] for f in r.iterrows()]
vectors = np.array([euler2vectors(f, rot_order, True) for f in euler])
vec_df.insert(loc=0, column='%s_xUp'%joint, value=pd.Series(data=[e[0] for e in vectors], index=vec_df.index))
vec_df.insert(loc=0, column='%s_yUp'%joint, value=pd.Series(data=[e[1] for e in vectors], index=vec_df.index))
vec_df.insert(loc=0, column='%s_zUp'%joint, value=pd.Series(data=[e[2] for e in vectors], index=vec_df.index))
vec_df.insert(loc=0, column='%s_xFwd'%joint, value=pd.Series(data=[e[3] for e in vectors], index=vec_df.index))
vec_df.insert(loc=0, column='%s_yFwd'%joint, value=pd.Series(data=[e[4] for e in vectors], index=vec_df.index))
vec_df.insert(loc=0, column='%s_zFwd'%joint, value=pd.Series(data=[e[5] for e in vectors], index=vec_df.index))
#print(exp_df.columns)
new_track = track.clone()
new_track.values = vec_df
Q.append(new_track)
return Q
def _vectors_to_euler(self, X):
'''Converts Up and Fwd vectors to Euler angles'''
Q = []
for track in X:
channels = []
titles = []
vec_df = track.values
# Create a new DataFrame to store the exponential map rep
#euler_df = pd.DataFrame(index=exp_df.index)
euler_df = vec_df.copy()
# List the columns that contain rotation channels
vec_params = [c for c in vec_df.columns if ( any(p in c for p in ['xUp', 'yUp','zUp','xFwd', 'yFwd','zFwd']) and 'Nub' not in c)]
# List the joints that are not end sites, i.e., have channels
joints = (joint for joint in track.skeleton if 'Nub' not in joint)
for joint in joints:
r = vec_df[[c for c in vec_params if joint in c]] # Get the columns that belong to this joint
euler_df.drop(['%s_xUp'%joint, '%s_yUp'%joint, '%s_zUp'%joint, '%s_xFwd'%joint, '%s_yFwd'%joint, '%s_zFwd'%joint], axis=1, inplace=True)
vectors = [[f[1]['%s_xUp'%joint], f[1]['%s_yUp'%joint], f[1]['%s_zUp'%joint], f[1]['%s_xFwd'%joint], f[1]['%s_yFwd'%joint], f[1]['%s_zFwd'%joint]] for f in r.iterrows()] # Make sure the columsn are organized in xyz order
rot_order = track.skeleton[joint]['order']
euler_rots = [vectors2euler(f, rot_order, True) for f in vectors]
# Create the corresponding columns in the new DataFrame
euler_df['%s_%srotation'%(joint, rot_order[0])] = pd.Series(data=[e[0] for e in euler_rots], index=euler_df.index)
euler_df['%s_%srotation'%(joint, rot_order[1])] = pd.Series(data=[e[1] for e in euler_rots], index=euler_df.index)
euler_df['%s_%srotation'%(joint, rot_order[2])] = pd.Series(data=[e[2] for e in euler_rots], index=euler_df.index)
new_track = track.clone()
new_track.values = euler_df
Q.append(new_track)
return Q
class Mirror(BaseEstimator, TransformerMixin):
def __init__(self, axis="X", append=True):
"""
Mirrors the data
"""
self.axis = axis
self.append = append
def fit(self, X, y=None):
return self
def transform(self, X, y=None):
#print("Mirror: " + self.axis)
Q = []
if self.append:
for track in X:
Q.append(track)
for track in X:
channels = []
titles = []
if self.axis == "X":
signs = np.array([1,-1,-1])
if self.axis == "Y":
signs = np.array([-1,1,-1])
if self.axis == "Z":
signs = np.array([-1,-1,1])
euler_df = track.values
# Create a new DataFrame to store the exponential map rep
new_df = pd.DataFrame(index=euler_df.index)
# Copy the root positions into the new DataFrame
rxp = '%s_Xposition'%track.root_name
ryp = '%s_Yposition'%track.root_name
rzp = '%s_Zposition'%track.root_name
new_df[rxp] = pd.Series(data=-signs[0]*euler_df[rxp], index=new_df.index)
new_df[ryp] = pd.Series(data=-signs[1]*euler_df[ryp], index=new_df.index)
new_df[rzp] = pd.Series(data=-signs[2]*euler_df[rzp], index=new_df.index)
# List the columns that contain rotation channels
rots = [c for c in euler_df.columns if ('rotation' in c and 'Nub' not in c)]
#lft_rots = [c for c in euler_df.columns if ('Left' in c and 'rotation' in c and 'Nub' not in c)]
#rgt_rots = [c for c in euler_df.columns if ('Right' in c and 'rotation' in c and 'Nub' not in c)]
lft_joints = (joint for joint in track.skeleton if 'Left' in joint and 'Nub' not in joint)
rgt_joints = (joint for joint in track.skeleton if 'Right' in joint and 'Nub' not in joint)
new_track = track.clone()
for lft_joint in lft_joints:
#lr = euler_df[[c for c in rots if lft_joint + "_" in c]]
#rot_order = track.skeleton[lft_joint]['order']
#lft_eulers = [[f[1]['%s_Xrotation'%lft_joint], f[1]['%s_Yrotation'%lft_joint], f[1]['%s_Zrotation'%lft_joint]] for f in lr.iterrows()]
rgt_joint = lft_joint.replace('Left', 'Right')
#rr = euler_df[[c for c in rots if rgt_joint + "_" in c]]
#rot_order = track.skeleton[rgt_joint]['order']
# rgt_eulers = [[f[1]['%s_Xrotation'%rgt_joint], f[1]['%s_Yrotation'%rgt_joint], f[1]['%s_Zrotation'%rgt_joint]] for f in rr.iterrows()]
# Create the corresponding columns in the new DataFrame
new_df['%s_Xrotation'%lft_joint] = pd.Series(data=signs[0]*track.values['%s_Xrotation'%rgt_joint], index=new_df.index)
new_df['%s_Yrotation'%lft_joint] = pd.Series(data=signs[1]*track.values['%s_Yrotation'%rgt_joint], index=new_df.index)
new_df['%s_Zrotation'%lft_joint] = pd.Series(data=signs[2]*track.values['%s_Zrotation'%rgt_joint], index=new_df.index)
new_df['%s_Xrotation'%rgt_joint] = pd.Series(data=signs[0]*track.values['%s_Xrotation'%lft_joint], index=new_df.index)
new_df['%s_Yrotation'%rgt_joint] = pd.Series(data=signs[1]*track.values['%s_Yrotation'%lft_joint], index=new_df.index)
new_df['%s_Zrotation'%rgt_joint] = pd.Series(data=signs[2]*track.values['%s_Zrotation'%lft_joint], index=new_df.index)
# List the joints that are not left or right, i.e. are on the trunk
joints = (joint for joint in track.skeleton if 'Nub' not in joint and 'Left' not in joint and 'Right' not in joint)
for joint in joints:
#r = euler_df[[c for c in rots if joint in c]] # Get the columns that belong to this joint
#rot_order = track.skeleton[joint]['order']
#eulers = [[f[1]['%s_Xrotation'%joint], f[1]['%s_Yrotation'%joint], f[1]['%s_Zrotation'%joint]] for f in r.iterrows()]
# Create the corresponding columns in the new DataFrame
new_df['%s_Xrotation'%joint] = pd.Series(data=signs[0]*track.values['%s_Xrotation'%joint], index=new_df.index)
new_df['%s_Yrotation'%joint] = pd.Series(data=signs[1]*track.values['%s_Yrotation'%joint], index=new_df.index)
new_df['%s_Zrotation'%joint] = pd.Series(data=signs[2]*track.values['%s_Zrotation'%joint], index=new_df.index)
new_track.values = new_df
new_track.take_name = track.take_name + "_mirrored"
Q.append(new_track)
return Q
def inverse_transform(self, X, copy=None, start_pos=None):
return X
class EulerReorder(BaseEstimator, TransformerMixin):
def __init__(self, new_order):
"""
Add a
"""
self.new_order = new_order
def fit(self, X, y=None):
self.orig_skeleton = copy.deepcopy(X[0].skeleton)
return self
def transform(self, X, y=None):
#print("EulerReorder")
Q = []
for track in X:
channels = []
titles = []
euler_df = track.values
# Create a new DataFrame to store the exponential map rep
#new_df = pd.DataFrame(index=euler_df.index)
new_df = euler_df.copy()
# Copy the root positions into the new DataFrame
rxp = '%s_Xposition'%track.root_name
ryp = '%s_Yposition'%track.root_name
rzp = '%s_Zposition'%track.root_name
new_df[rxp] = pd.Series(data=euler_df[rxp], index=new_df.index)
new_df[ryp] = pd.Series(data=euler_df[ryp], index=new_df.index)
new_df[rzp] = pd.Series(data=euler_df[rzp], index=new_df.index)
# List the columns that contain rotation channels
rots = [c for c in euler_df.columns if ('rotation' in c and 'Nub' not in c)]
# List the joints that are not end sites, i.e., have channels
joints = (joint for joint in track.skeleton if 'Nub' not in joint)
new_track = track.clone()
for joint in joints:
r = euler_df[[c for c in rots if joint in c]] # Get the columns that belong to this joint
rot_order = track.skeleton[joint]['order']
r1_col = '%s_%srotation'%(joint, rot_order[0])
r2_col = '%s_%srotation'%(joint, rot_order[1])
r3_col = '%s_%srotation'%(joint, rot_order[2])
#euler = [[f[1][r1_col], f[1][r2_col], f[1][r3_col]] for f in r.iterrows()]
euler = np.transpose(np.array([r[r1_col], r[r2_col], r[r3_col]]))
#euler = [[f[1]['%s_Xrotation'%(joint)], f[1]['%s_Yrotation'%(joint)], f[1]['%s_Zrotation'%(joint)]] for f in r.iterrows()]
new_euler = [euler_reorder(f, rot_order, self.new_order, True) for f in euler]
#new_euler = euler_reorder2(np.array(euler), rot_order, self.new_order, True)
# Create the corresponding columns in the new DataFrame
new_df['%s_%srotation'%(joint, self.new_order[0])] = pd.Series(data=[e[0] for e in new_euler], index=new_df.index)
new_df['%s_%srotation'%(joint, self.new_order[1])] = pd.Series(data=[e[1] for e in new_euler], index=new_df.index)
new_df['%s_%srotation'%(joint, self.new_order[2])] = pd.Series(data=[e[2] for e in new_euler], index=new_df.index)
new_track.skeleton[joint]['order'] = self.new_order
new_track.values = new_df
Q.append(new_track)
return Q
def inverse_transform(self, X, copy=None, start_pos=None):
return X
class JointSelector(BaseEstimator, TransformerMixin):
'''
Allows for filtering the mocap data to include only the selected joints
'''
def __init__(self, joints, include_root=False):
self.joints = joints
self.include_root = include_root
def fit(self, X, y=None):
selected_joints = []
selected_channels = []
if self.include_root:
selected_joints.append(X[0].root_name)
selected_joints.extend(self.joints)
for joint_name in selected_joints:
if joint_name.endswith("_Nub"):
selected_channels.extend([o for o in X[0].values.columns if (joint_name + "_") in o])
else:
selected_channels.extend([o for o in X[0].values.columns if (joint_name + "_") in o and 'Nub' not in o])
self.selected_joints = selected_joints
self.selected_channels = selected_channels
self.not_selected = X[0].values.columns.difference(selected_channels)
self.not_selected_values = {c:X[0].values[c].values[0] for c in self.not_selected}
self.orig_skeleton = X[0].skeleton
return self
def transform(self, X, y=None):
#print("JointSelector")
Q = []
for track in X:
t2 = track.clone()
for key in track.skeleton.keys():
if key not in self.selected_joints:
t2.skeleton.pop(key)
t2.values = track.values[self.selected_channels]
for key in t2.skeleton.keys():
to_remove = list(set(t2.skeleton[key]['children']) - set(self.selected_joints))
[t2.skeleton[key]['children'].remove(c) for c in to_remove]
Q.append(t2)
return Q
def inverse_transform(self, X, copy=None):
Q = []
for track in X:
t2 = track.clone()
skeleton = self.orig_skeleton
for key in track.skeleton.keys():
skeleton[key]['order']=track.skeleton[key]['order']
t2.skeleton = skeleton
for d in self.not_selected:
t2.values[d] = self.not_selected_values[d]
Q.append(t2)
return Q
class Numpyfier(BaseEstimator, TransformerMixin):
'''
Just converts the values in a MocapData object into a numpy array
Useful for the final stage of a pipeline before training
'''
def __init__(self):
pass
def fit(self, X, y=None):
self.org_mocap_ = X[0].clone()
self.org_mocap_.values.drop(self.org_mocap_.values.index, inplace=True)
return self
def transform(self, X, y=None):
#print("Numpyfier")
Q = []
for track in X:
Q.append(track.values.values)
#print("Numpyfier:" + str(track.values.columns))
return np.array(Q)
def inverse_transform(self, X, copy=None):
Q = []
for track in X:
new_mocap = self.org_mocap_.clone()
time_index = pd.to_timedelta([f for f in range(track.shape[0])], unit='s')*self.org_mocap_.framerate
new_df = pd.DataFrame(data=track, index=time_index, columns=self.org_mocap_.values.columns)
new_mocap.values = new_df
Q.append(new_mocap)
return Q
class Slicer(BaseEstimator, TransformerMixin):
'''
Slice the data into intervals of equal size
'''
def __init__(self, window_size, overlap=0.5):
self.window_size = window_size
self.overlap = overlap
pass
def fit(self, X, y=None):
self.org_mocap_ = X[0].clone()
self.org_mocap_.values.drop(self.org_mocap_.values.index, inplace=True)
return self
def transform(self, X, y=None):
#print("Slicer")
Q = []
for track in X:
vals = track.values.values
nframes = vals.shape[0]
overlap_frames = (int)(self.overlap*self.window_size)
n_sequences = (nframes-overlap_frames)//(self.window_size-overlap_frames)
if n_sequences>0:
y = np.zeros((n_sequences, self.window_size, vals.shape[1]))
# extract sequences from the input data
for i in range(0,n_sequences):
frameIdx = (self.window_size-overlap_frames) * i
Q.append(vals[frameIdx:frameIdx+self.window_size,:])
return np.array(Q)
def inverse_transform(self, X, copy=None):
Q = []
for track in X:
new_mocap = self.org_mocap_.clone()
time_index = pd.to_timedelta([f for f in range(track.shape[0])], unit='s')
new_df = pd.DataFrame(data=track, index=time_index, columns=self.org_mocap_.values.columns)
new_mocap.values = new_df
Q.append(new_mocap)
return Q
class RootTransformer(BaseEstimator, TransformerMixin):
def __init__(self, method, hips_axis_order="XYZ", position_smoothing=0, rotation_smoothing=0, separate_root=True):
"""
Accepted methods:
abdolute_translation_deltas
pos_rot_deltas
"""
self.method = method
self.position_smoothing=position_smoothing
self.rotation_smoothing=rotation_smoothing
self.separate_root = separate_root
self.hips_axis_order = hips_axis_order
# relative rotation from the hips awis the the x-side, y-up, z-forward convention
rot_mat = np.zeros((3,3))
for i in range(3):
ax_i = ord(hips_axis_order[i])-ord("X")
rot_mat[i,ax_i]=1
self.root_rotation_offset = Quaternions.from_transforms(rot_mat[np.newaxis, :, :])
self.hips_side_axis = -rot_mat[0,:]
def fit(self, X, y=None):
return self
def transform(self, X, y=None):
#print("RootTransformer")
Q = []
for track in X:
if self.method == 'abdolute_translation_deltas':
new_df = track.values.copy()
xpcol = '%s_Xposition'%track.root_name
ypcol = '%s_Yposition'%track.root_name
zpcol = '%s_Zposition'%track.root_name
dxpcol = '%s_dXposition'%track.root_name
dzpcol = '%s_dZposition'%track.root_name
x=track.values[xpcol].copy()
z=track.values[zpcol].copy()
if self.position_smoothing>0:
x_sm = filters.gaussian_filter1d(x, self.position_smoothing, axis=0, mode='nearest')
z_sm = filters.gaussian_filter1d(z, self.position_smoothing, axis=0, mode='nearest')
dx = pd.Series(data=x_sm, index=new_df.index).diff()
dz = pd.Series(data=z_sm, index=new_df.index).diff()
new_df[xpcol] = x-x_sm
new_df[zpcol] = z-z_sm
else:
dx = x.diff()
dz = z.diff()
new_df.drop([xpcol, zpcol], axis=1, inplace=True)
dx[0] = dx[1]
dz[0] = dz[1]
new_df[dxpcol] = dx
new_df[dzpcol] = dz
new_track = track.clone()
new_track.values = new_df
# end of abdolute_translation_deltas
elif self.method == 'pos_rot_deltas':
new_track = track.clone()
# Absolute columns
xp_col = '%s_Xposition'%track.root_name
yp_col = '%s_Yposition'%track.root_name
zp_col = '%s_Zposition'%track.root_name
#rot_order = track.skeleton[track.root_name]['order']
#%(joint, rot_order[0])
rot_order = track.skeleton[track.root_name]['order']
r1_col = '%s_%srotation'%(track.root_name, rot_order[0])
r2_col = '%s_%srotation'%(track.root_name, rot_order[1])
r3_col = '%s_%srotation'%(track.root_name, rot_order[2])
# Delta columns
# dxp_col = '%s_dXposition'%track.root_name
# dzp_col = '%s_dZposition'%track.root_name
# dxr_col = '%s_dXrotation'%track.root_name
# dyr_col = '%s_dYrotation'%track.root_name
# dzr_col = '%s_dZrotation'%track.root_name
dxp_col = 'reference_dXposition'
dzp_col = 'reference_dZposition'
dxr_col = 'reference_dXrotation'
dyr_col = 'reference_dYrotation'
dzr_col = 'reference_dZrotation'
positions = np.transpose(np.array([track.values[xp_col], track.values[yp_col], track.values[zp_col]]))
rotations = np.pi/180.0*np.transpose(np.array([track.values[r1_col], track.values[r2_col], track.values[r3_col]]))
""" Get Trajectory and smooth it"""
trajectory_filterwidth = self.position_smoothing
reference = positions.copy()*np.array([1,0,1])
if trajectory_filterwidth>0:
reference = filters.gaussian_filter1d(reference, trajectory_filterwidth, axis=0, mode='nearest')
""" Get Root Velocity """
velocity = np.diff(reference, axis=0)
velocity = np.vstack((velocity[0,:], velocity))
""" Remove Root Translation """
positions = positions-reference
""" Get Forward Direction along the x-z plane, assuming character is facig z-forward """
#forward = [Rotation(f, 'euler', from_deg=True, order=rot_order).rotmat[:,2] for f in rotations] # get the z-axis of the rotation matrix, assuming character is facig z-forward
#print("order:" + rot_order.lower())
quats = Quaternions.from_euler(rotations, order=rot_order.lower(), world=False)
#forward = quats*np.array([[0,0,1]])
#forward[:,1] = 0
side_dirs = quats*self.hips_side_axis
forward = np.cross(np.array([[0,1,0]]), side_dirs)
""" Smooth Forward Direction """
direction_filterwidth = self.rotation_smoothing
if direction_filterwidth>0:
forward = filters.gaussian_filter1d(forward, direction_filterwidth, axis=0, mode='nearest')
forward = forward / np.sqrt((forward**2).sum(axis=-1))[...,np.newaxis]
""" Remove Y Rotation """
target = np.array([[0,0,1]]).repeat(len(forward), axis=0)
rotation = Quaternions.between(target, forward)[:,np.newaxis]
positions = (-rotation[:,0]) * positions
#new_rotations = (-rotation[:,0]) * quats
new_rotations = (-self.root_rotation_offset) * (-rotation[:,0]) * quats
""" Get Root Rotation """
#print(rotation[:,0])
velocity = (-rotation[:,0]) * velocity
rvelocity = Pivots.from_quaternions(rotation[1:] * -rotation[:-1]).ps
rvelocity = np.vstack((rvelocity[0], rvelocity))
eulers = np.array([t3d.euler.quat2euler(q, axes=('s'+rot_order.lower()[::-1]))[::-1] for q in new_rotations])*180.0/np.pi
new_df = track.values.copy()
root_pos_x = pd.Series(data=positions[:,0], index=new_df.index)
root_pos_y = pd.Series(data=positions[:,1], index=new_df.index)
root_pos_z = pd.Series(data=positions[:,2], index=new_df.index)
root_pos_x_diff = pd.Series(data=velocity[:,0], index=new_df.index)
root_pos_z_diff = pd.Series(data=velocity[:,2], index=new_df.index)
root_rot_1 = pd.Series(data=eulers[:,0], index=new_df.index)
root_rot_2 = pd.Series(data=eulers[:,1], index=new_df.index)
root_rot_3 = pd.Series(data=eulers[:,2], index=new_df.index)
root_rot_y_diff = pd.Series(data=rvelocity[:,0], index=new_df.index)
#new_df.drop([xr_col, yr_col, zr_col, xp_col, zp_col], axis=1, inplace=True)
new_df[xp_col] = root_pos_x
new_df[yp_col] = root_pos_y
new_df[zp_col] = root_pos_z
new_df[dxp_col] = root_pos_x_diff
new_df[dzp_col] = root_pos_z_diff
new_df[r1_col] = root_rot_1
new_df[r2_col] = root_rot_2
new_df[r3_col] = root_rot_3
#new_df[dxr_col] = root_rot_x_diff
new_df[dyr_col] = root_rot_y_diff
#new_df[dzr_col] = root_rot_z_diff
new_track.values = new_df
elif self.method == 'pos_xyz_rot_deltas':
new_track = track.clone()
# Absolute columns
xp_col = '%s_Xposition'%track.root_name
yp_col = '%s_Yposition'%track.root_name
zp_col = '%s_Zposition'%track.root_name
#rot_order = track.skeleton[track.root_name]['order']
#%(joint, rot_order[0])
rot_order = track.skeleton[track.root_name]['order']
r1_col = '%s_%srotation'%(track.root_name, rot_order[0])
r2_col = '%s_%srotation'%(track.root_name, rot_order[1])
r3_col = '%s_%srotation'%(track.root_name, rot_order[2])
# Delta columns
# dxp_col = '%s_dXposition'%track.root_name
# dzp_col = '%s_dZposition'%track.root_name
# dxr_col = '%s_dXrotation'%track.root_name
# dyr_col = '%s_dYrotation'%track.root_name
# dzr_col = '%s_dZrotation'%track.root_name
dxp_col = 'reference_dXposition'
dyp_col = 'reference_dYposition'
dzp_col = 'reference_dZposition'
dxr_col = 'reference_dXrotation'
dyr_col = 'reference_dYrotation'
dzr_col = 'reference_dZrotation'
positions = np.transpose(np.array([track.values[xp_col], track.values[yp_col], track.values[zp_col]]))
rotations = np.pi/180.0*np.transpose(np.array([track.values[r1_col], track.values[r2_col], track.values[r3_col]]))
""" Get Trajectory and smooth it"""
trajectory_filterwidth = self.position_smoothing
#reference = positions.copy()*np.array([1,0,1])
if trajectory_filterwidth>0:
reference = filters.gaussian_filter1d(positions, trajectory_filterwidth, axis=0, mode='nearest')
""" Get Root Velocity """
velocity = np.diff(reference, axis=0)
velocity = np.vstack((velocity[0,:], velocity))
""" Remove Root Translation """
positions = positions-reference
""" Get Forward Direction along the x-z plane, assuming character is facig z-forward """
#forward = [Rotation(f, 'euler', from_deg=True, order=rot_order).rotmat[:,2] for f in rotations] # get the z-axis of the rotation matrix, assuming character is facig z-forward
#print("order:" + rot_order.lower())
quats = Quaternions.from_euler(rotations, order=rot_order.lower(), world=False)
#calculate the hips forward directions given in global cordinates
#side_ax = np.zeros((1,3))
#side_ax[0,self.hips_side_axis]=1
#side_dirs = quats*side_ax
side_dirs = quats*self.hips_side_axis
forward = np.cross(np.array([[0,1,0]]), side_dirs)
""" Smooth Forward Direction """
direction_filterwidth = self.rotation_smoothing
if direction_filterwidth>0:
forward = filters.gaussian_filter1d(forward, direction_filterwidth, axis=0, mode='nearest')
# make unit vector
forward = forward / np.sqrt((forward**2).sum(axis=-1))[...,np.newaxis]
""" Remove Y Rotation """
target = np.array([[0,0,1]]).repeat(len(forward), axis=0)
rotation = Quaternions.between(target, forward)[:,np.newaxis]
positions = (-rotation[:,0]) * positions
new_rotations = (-self.root_rotation_offset) * (-rotation[:,0]) * quats
""" Get Root Rotation """
#print(rotation[:,0])
velocity = (-rotation[:,0]) * velocity
rvelocity = Pivots.from_quaternions(rotation[1:] * -rotation[:-1]).ps
rvelocity = np.vstack((rvelocity[0], rvelocity))
eulers = np.array([t3d.euler.quat2euler(q, axes=('s'+rot_order.lower()[::-1]))[::-1] for q in new_rotations])*180.0/np.pi
new_df = track.values.copy()
root_pos_x = pd.Series(data=positions[:,0], index=new_df.index)
root_pos_y = pd.Series(data=positions[:,1], index=new_df.index)
root_pos_z = pd.Series(data=positions[:,2], index=new_df.index)
root_pos_x_diff = pd.Series(data=velocity[:,0], index=new_df.index)
root_pos_y_diff = pd.Series(data=velocity[:,1], index=new_df.index)
root_pos_z_diff = pd.Series(data=velocity[:,2], index=new_df.index)
root_rot_1 = pd.Series(data=eulers[:,0], index=new_df.index)
root_rot_2 = pd.Series(data=eulers[:,1], index=new_df.index)
root_rot_3 = pd.Series(data=eulers[:,2], index=new_df.index)
root_rot_y_diff = pd.Series(data=rvelocity[:,0], index=new_df.index)
#new_df.drop([xr_col, yr_col, zr_col, xp_col, zp_col], axis=1, inplace=True)
new_df[xp_col] = root_pos_x
new_df[yp_col] = root_pos_y
new_df[zp_col] = root_pos_z
new_df[dxp_col] = root_pos_x_diff
new_df[dyp_col] = root_pos_y_diff
new_df[dzp_col] = root_pos_z_diff
new_df[r1_col] = root_rot_1
new_df[r2_col] = root_rot_2
new_df[r3_col] = root_rot_3
#new_df[dxr_col] = root_rot_x_diff
new_df[dyr_col] = root_rot_y_diff
#new_df[dzr_col] = root_rot_z_diff
new_track.values = new_df
elif self.method == 'hip_centric':
new_track = track.clone()
# Absolute columns
xp_col = '%s_Xposition'%track.root_name
yp_col = '%s_Yposition'%track.root_name
zp_col = '%s_Zposition'%track.root_name
xr_col = '%s_Xrotation'%track.root_name
yr_col = '%s_Yrotation'%track.root_name
zr_col = '%s_Zrotation'%track.root_name
new_df = track.values.copy()
all_zeros = np.zeros(track.values[xp_col].values.shape)
new_df[xp_col] = pd.Series(data=all_zeros, index=new_df.index)
new_df[yp_col] = pd.Series(data=all_zeros, index=new_df.index)
new_df[zp_col] = pd.Series(data=all_zeros, index=new_df.index)
new_df[zp_col] = pd.Series(data=all_zeros, index=new_df.index)
new_df[xr_col] = pd.Series(data=all_zeros, index=new_df.index)
new_df[yr_col] = pd.Series(data=all_zeros, index=new_df.index)
new_df[zr_col] = pd.Series(data=all_zeros, index=new_df.index)
new_track.values = new_df
#print(new_track.values.columns)
Q.append(new_track)
return Q
def inverse_transform(self, X, copy=None, start_pos=None):
Q = []
#TODO: simplify this implementation
startx = 0
startz = 0
if start_pos is not None:
startx, startz = start_pos
for track in X:
new_track = track.clone()
if self.method == 'abdolute_translation_deltas':
new_df = new_track.values
xpcol = '%s_Xposition'%track.root_name
ypcol = '%s_Yposition'%track.root_name
zpcol = '%s_Zposition'%track.root_name
dxpcol = '%s_dXposition'%track.root_name
dzpcol = '%s_dZposition'%track.root_name
dx = track.values[dxpcol].values
dz = track.values[dzpcol].values
recx = [startx]
recz = [startz]
for i in range(dx.shape[0]-1):
recx.append(recx[i]+dx[i+1])
recz.append(recz[i]+dz[i+1])
# recx = [recx[i]+dx[i+1] for i in range(dx.shape[0]-1)]
# recz = [recz[i]+dz[i+1] for i in range(dz.shape[0]-1)]
# recx = dx[:-1] + dx[1:]
# recz = dz[:-1] + dz[1:]
if self.position_smoothing > 0:
new_df[xpcol] = pd.Series(data=new_df[xpcol]+recx, index=new_df.index)
new_df[zpcol] = pd.Series(data=new_df[zpcol]+recz, index=new_df.index)
else:
new_df[xpcol] = pd.Series(data=recx, index=new_df.index)
new_df[zpcol] = pd.Series(data=recz, index=new_df.index)
new_df.drop([dxpcol, dzpcol], axis=1, inplace=True)
new_track.values = new_df
# end of abdolute_translation_deltas
elif self.method == 'pos_rot_deltas':
# Absolute columns
rot_order = track.skeleton[track.root_name]['order']
xp_col = '%s_Xposition'%track.root_name
yp_col = '%s_Yposition'%track.root_name
zp_col = '%s_Zposition'%track.root_name
xr_col = '%s_Xrotation'%track.root_name
yr_col = '%s_Yrotation'%track.root_name
zr_col = '%s_Zrotation'%track.root_name
r1_col = '%s_%srotation'%(track.root_name, rot_order[0])
r2_col = '%s_%srotation'%(track.root_name, rot_order[1])
r3_col = '%s_%srotation'%(track.root_name, rot_order[2])
# Delta columns
# dxp_col = '%s_dXposition'%track.root_name
# dzp_col = '%s_dZposition'%track.root_name
# dyr_col = '%s_dYrotation'%track.root_name
dxp_col = 'reference_dXposition'
dzp_col = 'reference_dZposition'
dyr_col = 'reference_dYrotation'
positions = np.transpose(np.array([track.values[xp_col], track.values[yp_col], track.values[zp_col]]))
rotations = np.pi/180.0*np.transpose(np.array([track.values[r1_col], track.values[r2_col], track.values[r3_col]]))
quats = Quaternions.from_euler(rotations, order=rot_order.lower(), world=False)
new_df = track.values.copy()
dx = track.values[dxp_col].values
dz = track.values[dzp_col].values
dry = track.values[dyr_col].values
#rec_p = np.array([startx, 0, startz])+positions[0,:]
rec_ry = Quaternions.id(quats.shape[0])
rec_xp = [0]
rec_zp = [0]
#rec_r = Quaternions.id(quats.shape[0])
for i in range(dx.shape[0]-1):
#print(dry[i])
q_y = Quaternions.from_angle_axis(np.array(dry[i+1]), np.array([0,1,0]))
rec_ry[i+1] = q_y*rec_ry[i]
#print("dx: + " + str(dx[i+1]))
dp = rec_ry[i+1]*np.array([dx[i+1], 0, dz[i+1]])
rec_xp.append(rec_xp[i]+dp[0,0])
rec_zp.append(rec_zp[i]+dp[0,2])
if self.separate_root:
qq = quats
xx = positions[:,0]
zz = positions[:,2]
else:
qq = rec_ry*self.root_rotation_offset*quats
pp = rec_ry*positions
xx = rec_xp + pp[:,0]
zz = rec_zp + pp[:,2]
eulers = np.array([t3d.euler.quat2euler(q, axes=('s'+rot_order.lower()[::-1]))[::-1] for q in qq])*180.0/np.pi
new_df = track.values.copy()
root_rot_1 = pd.Series(data=eulers[:,0], index=new_df.index)
root_rot_2 = pd.Series(data=eulers[:,1], index=new_df.index)
root_rot_3 = pd.Series(data=eulers[:,2], index=new_df.index)
new_df[xp_col] = pd.Series(data=xx, index=new_df.index)
new_df[zp_col] = pd.Series(data=zz, index=new_df.index)
new_df[r1_col] = pd.Series(data=root_rot_1, index=new_df.index)
new_df[r2_col] = pd.Series(data=root_rot_2, index=new_df.index)
new_df[r3_col] = pd.Series(data=root_rot_3, index=new_df.index)
if self.separate_root:
ref_rot_order="ZXY"
new_df["reference_Xposition"] = pd.Series(data=rec_xp, index=new_df.index)
new_df["reference_Zposition"] = pd.Series(data=rec_zp, index=new_df.index)
eulers_ry = np.array([t3d.euler.quat2euler(q, axes=('s'+ref_rot_order.lower()[::-1]))[::-1] for q in rec_ry])*180.0/np.pi
new_df["reference_Yrotation"] = pd.Series(data=eulers_ry[:,ref_rot_order.find('Y')], index=new_df.index)
new_df.drop([dyr_col, dxp_col, dzp_col], axis=1, inplace=True)
new_track.values = new_df
elif self.method == 'pos_xyz_rot_deltas':
# Absolute columns
rot_order = track.skeleton[track.root_name]['order']
xp_col = '%s_Xposition'%track.root_name
yp_col = '%s_Yposition'%track.root_name
zp_col = '%s_Zposition'%track.root_name
xr_col = '%s_Xrotation'%track.root_name
yr_col = '%s_Yrotation'%track.root_name
zr_col = '%s_Zrotation'%track.root_name
r1_col = '%s_%srotation'%(track.root_name, rot_order[0])
r2_col = '%s_%srotation'%(track.root_name, rot_order[1])
r3_col = '%s_%srotation'%(track.root_name, rot_order[2])
# Delta columns
# dxp_col = '%s_dXposition'%track.root_name
# dzp_col = '%s_dZposition'%track.root_name
# dyr_col = '%s_dYrotation'%track.root_name
dxp_col = 'reference_dXposition'
dyp_col = 'reference_dYposition'
dzp_col = 'reference_dZposition'
dyr_col = 'reference_dYrotation'
positions = np.transpose(np.array([track.values[xp_col], track.values[yp_col], track.values[zp_col]]))
rotations = np.pi/180.0*np.transpose(np.array([track.values[r1_col], track.values[r2_col], track.values[r3_col]]))
quats = Quaternions.from_euler(rotations, order=rot_order.lower(), world=False)
new_df = track.values.copy()
dx = track.values[dxp_col].values
dy = track.values[dyp_col].values
dz = track.values[dzp_col].values
dry = track.values[dyr_col].values
#rec_p = np.array([startx, 0, startz])+positions[0,:]
rec_ry = Quaternions.id(quats.shape[0])
rec_xp = [0]
rec_yp = [0]
rec_zp = [0]
#rec_r = Quaternions.id(quats.shape[0])
for i in range(dx.shape[0]-1):
#print(dry[i])
q_y = Quaternions.from_angle_axis(np.array(dry[i+1]), np.array([0,1,0]))
rec_ry[i+1] = q_y*rec_ry[i]
#print("dx: + " + str(dx[i+1]))
dp = rec_ry[i+1]*np.array([dx[i+1], dy[i+1], dz[i+1]])
rec_xp.append(rec_xp[i]+dp[0,0])
rec_yp.append(rec_yp[i]+dp[0,1])
rec_zp.append(rec_zp[i]+dp[0,2])
if self.separate_root:
qq = quats
xx = positions[:,0]
yy = positions[:,1]
zz = positions[:,2]
else:
qq = rec_ry*self.root_rotation_offset*quats
pp = rec_ry*positions
xx = rec_xp + pp[:,0]
yy = rec_yp + pp[:,1]
zz = rec_zp + pp[:,2]
eulers = np.array([t3d.euler.quat2euler(q, axes=('s'+rot_order.lower()[::-1]))[::-1] for q in qq])*180.0/np.pi
new_df = track.values.copy()
root_rot_1 = pd.Series(data=eulers[:,0], index=new_df.index)
root_rot_2 = pd.Series(data=eulers[:,1], index=new_df.index)
root_rot_3 = pd.Series(data=eulers[:,2], index=new_df.index)
new_df[xp_col] = pd.Series(data=xx, index=new_df.index)
new_df[yp_col] = pd.Series(data=yy, index=new_df.index)
new_df[zp_col] = pd.Series(data=zz, index=new_df.index)
new_df[r1_col] = pd.Series(data=root_rot_1, index=new_df.index)
new_df[r2_col] = pd.Series(data=root_rot_2, index=new_df.index)
new_df[r3_col] = pd.Series(data=root_rot_3, index=new_df.index)
if self.separate_root:
new_df["reference_Xposition"] = pd.Series(data=rec_xp, index=new_df.index)
new_df["reference_Yposition"] = pd.Series(data=rec_yp, index=new_df.index)
new_df["reference_Zposition"] = pd.Series(data=rec_zp, index=new_df.index)
eulers_ry = np.array([t3d.euler.quat2euler(q, axes=('s'+rot_order.lower()[::-1]))[::-1] for q in rec_ry])*180.0/np.pi
new_df["reference_Yrotation"] = pd.Series(data=eulers_ry[:,rot_order.find('Y')], index=new_df.index)
new_df.drop([dyr_col, dxp_col, dyp_col, dzp_col], axis=1, inplace=True)
new_track.values = new_df
#print(new_track.values.columns)
Q.append(new_track)
return Q
class RootCentricPositionNormalizer(BaseEstimator, TransformerMixin):
def __init__(self):
pass
def fit(self, X, y=None):
return self
def transform(self, X, y=None):
Q = []
for track in X:
new_track = track.clone()
rxp = '%s_Xposition'%track.root_name
ryp = '%s_Yposition'%track.root_name
rzp = '%s_Zposition'%track.root_name
projected_root_pos = track.values[[rxp, ryp, rzp]]
projected_root_pos.loc[:,ryp] = 0 # we want the root's projection on the floor plane as the ref
new_df = pd.DataFrame(index=track.values.index)
all_but_root = [joint for joint in track.skeleton if track.root_name not in joint]
# all_but_root = [joint for joint in track.skeleton]
for joint in all_but_root:
new_df['%s_Xposition'%joint] = pd.Series(data=track.values['%s_Xposition'%joint]-projected_root_pos[rxp], index=new_df.index)
new_df['%s_Yposition'%joint] = pd.Series(data=track.values['%s_Yposition'%joint]-projected_root_pos[ryp], index=new_df.index)
new_df['%s_Zposition'%joint] = pd.Series(data=track.values['%s_Zposition'%joint]-projected_root_pos[rzp], index=new_df.index)
# keep the root as it is now
new_df[rxp] = track.values[rxp]
new_df[ryp] = track.values[ryp]
new_df[rzp] = track.values[rzp]
new_track.values = new_df
Q.append(new_track)
return Q
def inverse_transform(self, X, copy=None):
Q = []
for track in X:
new_track = track.clone()
rxp = '%s_Xposition'%track.root_name
ryp = '%s_Yposition'%track.root_name
rzp = '%s_Zposition'%track.root_name
projected_root_pos = track.values[[rxp, ryp, rzp]]
projected_root_pos.loc[:,ryp] = 0 # we want the root's projection on the floor plane as the ref
new_df = pd.DataFrame(index=track.values.index)
for joint in track.skeleton:
new_df['%s_Xposition'%joint] = pd.Series(data=track.values['%s_Xposition'%joint]+projected_root_pos[rxp], index=new_df.index)
new_df['%s_Yposition'%joint] = pd.Series(data=track.values['%s_Yposition'%joint]+projected_root_pos[ryp], index=new_df.index)
new_df['%s_Zposition'%joint] = pd.Series(data=track.values['%s_Zposition'%joint]+projected_root_pos[rzp], index=new_df.index)
new_track.values = new_df
Q.append(new_track)
return Q
class Flattener(BaseEstimator, TransformerMixin):
def __init__(self):
pass
def fit(self, X, y=None):
return self
def transform(self, X, y=None):
return np.concatenate(X, axis=0)
class ConstantsRemover(BaseEstimator, TransformerMixin):
'''
For now it just looks at the first track
'''
def __init__(self, eps = 1e-6):
self.eps = eps
def fit(self, X, y=None):
stds = X[0].values.std()
cols = X[0].values.columns.values
self.const_dims_ = [c for c in cols if (stds[c] < self.eps).any()]
self.const_values_ = {c:X[0].values[c].values[0] for c in cols if (stds[c] < self.eps).any()}
return self
def transform(self, X, y=None):
Q = []
for track in X:
t2 = track.clone()
#for key in t2.skeleton.keys():
# if key in self.ConstDims_:
# t2.skeleton.pop(key)
#print(track.values.columns.difference(self.const_dims_))
t2.values.drop(self.const_dims_, axis=1, inplace=True)
#t2.values = track.values[track.values.columns.difference(self.const_dims_)]
Q.append(t2)
return Q
def inverse_transform(self, X, copy=None):
Q = []
for track in X:
t2 = track.clone()
for d in self.const_dims_:
t2.values[d] = self.const_values_[d]
# t2.values.assign(d=pd.Series(data=self.const_values_[d], index = t2.values.index))
Q.append(t2)
return Q
class ListStandardScaler(BaseEstimator, TransformerMixin):
def __init__(self, is_DataFrame=False):
self.is_DataFrame = is_DataFrame
def fit(self, X, y=None):
if self.is_DataFrame:
X_train_flat = np.concatenate([m.values for m in X], axis=0)
else:
X_train_flat = np.concatenate([m for m in X], axis=0)
self.data_mean_ = np.mean(X_train_flat, axis=0)
self.data_std_ = np.std(X_train_flat, axis=0)
return self
def transform(self, X, y=None):
Q = []
for track in X:
if self.is_DataFrame:
normalized_track = track.copy()
normalized_track.values = (track.values - self.data_mean_) / self.data_std_
else:
normalized_track = (track - self.data_mean_) / self.data_std_
Q.append(normalized_track)
if self.is_DataFrame:
return Q
else:
return np.array(Q)
def inverse_transform(self, X, copy=None):
Q = []
for track in X:
if self.is_DataFrame:
unnormalized_track = track.copy()
unnormalized_track.values = (track.values * self.data_std_) + self.data_mean_
else:
unnormalized_track = (track * self.data_std_) + self.data_mean_
Q.append(unnormalized_track)
if self.is_DataFrame:
return Q
else:
return np.array(Q)
class ListMinMaxScaler(BaseEstimator, TransformerMixin):
def __init__(self, is_DataFrame=False):
self.is_DataFrame = is_DataFrame
def fit(self, X, y=None):
if self.is_DataFrame:
X_train_flat = np.concatenate([m.values for m in X], axis=0)
else:
X_train_flat = np.concatenate([m for m in X], axis=0)
self.data_max_ = np.max(X_train_flat, axis=0)
self.data_min_ = np.min(X_train_flat, axis=0)
return self
def transform(self, X, y=None):
Q = []
for track in X:
if self.is_DataFrame:
normalized_track = track.copy()
normalized_track.values = (track.values - self.data_min_) / (self.data_max_ - self.data_min_)
else:
normalized_track = (track - self.data_min_) / (self.data_max_ - self.data_min_)
Q.append(normalized_track)
if self.is_DataFrame:
return Q
else:
return np.array(Q)
def inverse_transform(self, X, copy=None):
Q = []
for track in X:
if self.is_DataFrame:
unnormalized_track = track.copy()
unnormalized_track.values = (track.values * (self.data_max_ - self.data_min_)) + self.data_min_
else:
unnormalized_track = (track * (self.data_max_ - self.data_min_)) + self.data_min_
Q.append(unnormalized_track)
if self.is_DataFrame:
return Q
else:
return np.array(Q)
class Resampler(BaseEstimator, TransformerMixin):
def __init__(self, fps, method='cubic'):
'''
Method to resample a pandas dataframe to a different framerate.
NOTE: Pandas resampling is quit unintuitive when resampling to odd framerates using interpolation.
Thus we do it in this complex way.
'''
self.tgt_frametime = 1.0/fps
self.method = method
def fit(self, X, y=None):
#print("Resampling to tgt_frametime: " + str(self.tgt_frametime))
self.orig_frametime=X[0].framerate
return self
def resample_dataframe(self, df, frametime, method='cubic'):
#Create a time index for the resampled data
rate = str(round(1.0e9*frametime))+'N'
time_index = df.resample(rate).indices
#reindex the old data. This will turn all non-matching indices to NAN
tmp = df.reindex(time_index)
#merge with the old data and sort
tmp = pd.concat([df, tmp]).sort_index()
#remove duplicate time indices. Then fill the NAN values using interpolation
tmp=tmp[~tmp.index.duplicated(keep='first')].interpolate(method=method)
#return the values using the resampled indices
return tmp.loc[list(time_index)]
def resample_df(self, df, new_frametime, old_frametime, mode='cubic'):
#Create a time index for the resampled data
data = df.values
nframes = data.shape[0]
nframes_new = round(nframes*old_frametime/new_frametime)
x = np.arange(0, nframes)/(nframes-1)
xnew = np.arange(0, nframes_new)/(nframes_new-1)
data_out = np.zeros((nframes_new, data.shape[1]))
for jj in range(data.shape[1]):
y = data[:,jj]
f = interpolate.interp1d(x, y, bounds_error=False, kind=mode, fill_value='extrapolate')
data_out[:,jj] = f(xnew)
time_index = pd.to_timedelta([f for f in range(xnew.shape[0])], unit='s')*new_frametime
out = pd.DataFrame(data=data_out, index=time_index, columns=df.columns)
#Scale root deltas to match new frame-rate
sc = nframes/nframes_new
rootdelta_cols = [c for c in df.columns if ('reference_d' in c)]
out[rootdelta_cols]*=sc
return out
# def resample_poly_df(self, df, new_frametime, old_frametime):
# old_fps = round(1/old_frametime)
# new_fps = round(1/new_frametime)
# lcm = np.lcm(old_fps, new_fps)
# up = lcm//old_fps
# down = lcm//new_fps
# new_vals = signal.resample_poly(df.values, up, down, padtype='line')
# time_index = pd.to_timedelta([f for f in range(new_vals.shape[0])], unit='s')*new_frametime
# new_df = pd.DataFrame(data=new_vals, index=time_index, columns=df.columns)
# return new_df
def transform(self, X, y=None):
Q = []
for track in X:
new_track = track.clone()
# if self.method=="resample_poly":
# new_track.values = self.resample_poly_df(track.values, self.tgt_frametime, track.framerate)
# else:
new_track.values = self.resample_df(track.values, self.tgt_frametime, track.framerate, self.method)
#new_track.values = self.resample_dataframe(track.values, self.tgt_frametime, method=self.method)
new_track.framerate = self.tgt_frametime
Q.append(new_track)
return Q
def inverse_transform(self, X, copy=None):
Q = []
for track in X:
new_track = track.clone()
#new_track.values = self.resample_dataframe(track.values, self.orig_frametime, method=self.method)
if self.method=="resample_poly":
new_track.values = self.resample_poly_df(track.values, self.orig_frametime, track.framerate)
else:
new_track.values = self.resample_df(track.values, self.orig_frametime, track.framerate, self.method)
new_track.framerate = self.orig_frametime
Q.append(new_track)
return Q
class DownSampler(BaseEstimator, TransformerMixin):
def __init__(self, tgt_fps, keep_all=False):
self.tgt_fps = tgt_fps
self.keep_all = keep_all
def fit(self, X, y=None):
return self
def transform(self, X, y=None):
Q = []
for track in X:
orig_fps=round(1.0/track.framerate)
rate = orig_fps//self.tgt_fps
if orig_fps%self.tgt_fps!=0:
print("error orig_fps (" + str(orig_fps) + ") is not dividable with tgt_fps (" + str(self.tgt_fps) + ")")
else:
print("downsampling with rate: " + str(rate))
#print(track.values.size)
for ii in range(0,rate):
new_track = track.clone()
if self.keep_all:
new_track.take_name = new_track.take_name + "_" + str(ii).zfill(2)
new_track.values = track.values[ii::rate].copy()
#print(new_track.values.size)
#new_track = track[0:-1:self.rate]
new_track.framerate = 1.0/self.tgt_fps
Q.append(new_track)
if not self.keep_all:
break
return Q
def inverse_transform(self, X, copy=None):
return X
class ReverseTime(BaseEstimator, TransformerMixin):
def __init__(self, append=True):
self.append = append
def fit(self, X, y=None):
return self
def transform(self, X, y=None):
#print("ReverseTime")
Q = []
if self.append:
for track in X:
Q.append(track)
for track in X:
new_track = track.clone()
new_track.values = track.values[-1::-1]
new_track.values.index=new_track.values.index[0]-new_track.values.index
Q.append(new_track)
return Q
def inverse_transform(self, X, copy=None):
return X
class ListFeatureUnion(BaseEstimator, TransformerMixin):
def __init__(self, processors):
self.processors = processors
def fit(self, X, y=None):
assert(y is None)
for proc in self.processors:
if isinstance(proc, Pipeline):
#Loop steps and run fit on each. This is necessary since
#running fit on a Pipeline runs fit_transform on all steps
#and not only fit.
for step in proc.steps:
step[1].fit(X)
else:
proc.fit(X)
return self
def transform(self, X, y=None):
assert(y is None)
#print("ListFeatureUnion")
Q = []
idx=0
for proc in self.processors:
Z = proc.transform(X)
if idx==0:
Q = Z
else:
assert(len(Q)==len(Z))
for idx2,track in enumerate(Z):
Q[idx2].values = pd.concat([Q[idx2].values,Z[idx2].values], axis=1)
idx += 1
return Q
def inverse_transform(self, X, y=None):
return X
class RollingStatsCalculator(BaseEstimator, TransformerMixin):
'''
Creates a causal mean and std filter with a rolling window of length win (based on using prev and current values)
'''
def __init__(self, win):
self.win = win
def fit(self, X, y=None):
return self
def transform(self, X, y=None):
#print("RollingStatsCalculator: " + str(self.win))
Q = []
for track in X:
new_track = track.clone()
mean_df = track.values.rolling(window=self.win).mean()
std_df = track.values.rolling(window=self.win).std()
# rolling.mean results in Nans in start seq. Here we fill these
win = min(self.win, new_track.values.shape[0])
for i in range(1,win):
mm=track.values[:i].rolling(window=i).mean()
ss=track.values[:i].rolling(window=i).std()
mean_df.iloc[i-1] = mm.iloc[i-1]
std_df.iloc[i-1] = ss.iloc[i-1]
std_df.iloc[0] = std_df.iloc[1]
# Append to
new_track.values=pd.concat([mean_df.add_suffix('_mean'), std_df.add_suffix('_std')], axis=1)
Q.append(new_track)
return Q
def inverse_transform(self, X, copy=None):
return X
class FeatureCounter(BaseEstimator, TransformerMixin):
def __init__(self):
pass
def fit(self, X, y=None):
self.n_features = len(X[0].values.columns)
return self
def transform(self, X, y=None):
return X
def inverse_transform(self, X, copy=None):
return X
#TODO: JointsSelector (x)
#TODO: SegmentMaker
#TODO: DynamicFeaturesAdder
#TODO: ShapeFeaturesAdder
#TODO: DataFrameNumpier (x)
class TemplateTransform(BaseEstimator, TransformerMixin):
def __init__(self):
pass
def fit(self, X, y=None):
return self
def transform(self, X, y=None):
return X
================================================
FILE: pymo/rotation_tools.py
================================================
'''
Tools for Manipulating and Converting 3D Rotations
By Omid Alemi
Created: June 12, 2017
Adapted from that matlab file...
'''
import math
import numpy as np
import transforms3d as t3d
from pymo.Quaternions import Quaternions
def deg2rad(x):
return x/180*math.pi
def rad2deg(x):
return x/math.pi*180
def unroll(rots):
new_rot = rots.copy()
ang0 = np.linalg.norm(rots, axis=1) + 1e-8
idx = np.where(ang0>np.pi)[0]
ax = rots/np.tile(ang0[:,None], (1,3))
ang1=ang0-2*np.pi
alt_rot = ax*np.tile(ang1[:,None], (1,3))
new_rot[idx] = alt_rot[idx]
return new_rot
def unroll_1(rots):
new_rots = rots.copy()
# Compute angles and alternative rotation angles
angs = np.linalg.norm(rots, axis=1)
alt_angs=2*np.pi-angs
#find discontinuities
d_angs = np.diff(angs, axis=0)
d_angs2 = alt_angs[1:]-angs[:-1]
swps = np.where(np.abs(d_angs2)<np.abs(d_angs))[0]
#reshape into intervals where we should unroll the rotations
isodd = swps.shape[0] % 2 == 1
if isodd:
swps = np.append(swps, rots.shape[0]-1)
intv = 1+swps.reshape((swps.shape[0]//2, 2))
for ii in range(intv.shape[0]):
new_ax = -rots[intv[ii,0]:intv[ii,1],:]/np.tile(angs[intv[ii,0]:intv[ii,1], None], (1,3))
new_angs = alt_angs[intv[ii,0]:intv[ii,1]]
new_rots[intv[ii,0]:intv[ii,1],:] = new_ax*np.tile(new_angs[:, None], (1,3))
return new_rots
def unroll_2(rots):
new_rots = rots.copy()
# Compute angles and alternative rotation angles
angs = np.linalg.norm(rots, axis=1)
dotprod = np.einsum('ij,ij->i', rots[:-1,:], rots[1:,:])
#ax = rots/np.tile(angs[:, None], (1,3))
#d_ax = np.linalg.norm(np.diff(ax, axis=0), axis=1)
alt_angs=2*np.pi-angs
#find discontinuities
d_angs = np.diff(angs, axis=0)
d_angs2 = alt_angs[1:]-angs[:-1]
# FIXME should check if dot product is <0 not norm d_ax
swps = np.where((dotprod<-1))[0]
#swps = np.where((np.abs(d_ax)>0.5))[0]
#swps = np.where(np.abs(d_angs2)<np.abs(d_angs))[0]
#reshape into intervals where we should unroll the rotations
isodd = swps.shape[0] % 2 == 1
if isodd:
swps=swps[:-1]
#swps = np.append(swps, rots.shape[0]-1)
intv = 1+swps.reshape((swps.shape[0]//2, 2))
for ii in range(intv.shape[0]):
new_ax = -rots[intv[ii,0]:intv[ii,1],:]/np.tile(angs[intv[ii,0]:intv[ii,1], None], (1,3))
new_angs = alt_angs[intv[ii,0]:intv[ii,1]]
new_rots[intv[ii,0]:intv[ii,1],:] = new_ax*np.tile(new_angs[:, None], (1,3))
return new_rots
def euler_reorder2(rots, order='XYZ', new_order='XYZ',use_deg=False):
if order==new_order:
return rots
if use_deg:
rots = np.deg2rad(rots)
quats = Quaternions.from_euler(rots, order=order.lower())
eul = quats.euler(order=new_order.lower())
if use_deg:
eul = np.rad2deg(eul)
return eul
def euler_reorder(rot, order='XYZ', new_order='XYZ',use_deg=False):
if order==new_order:
return rot
if use_deg:
rot = np.deg2rad(rot)
rotmat = t3d.euler.euler2mat(rot[0], rot[1], rot[2], 'r' + order.lower())
eul = t3d.euler.mat2euler(rotmat, 'r' + new_order.lower())
#quat = t3d.euler.euler2quat(rot[0], rot[1], rot[2], 'r' + order.lower())
#eul = t3d.euler.quat2euler(quat, 'r' + new_order.lower())
if use_deg:
eul = np.rad2deg(eul)
return eul
def offsets_inv(offset, rots, order='XYZ',use_deg=False):
if use_deg:
offset = np.deg2rad(offset)
rots = np.deg2rad(rots)
q0 = t3d.euler.euler2quat(rots[0], rots[1], rots[2], 'r' + order.lower())
q_off = t3d.euler.euler2quat(offset[0], offset[1], offset[2], 'r' + order.lower())
q2=t3d.euler.quat2euler(t3d.quaternions.qmult(q0,t3d.quaternions.qinverse(q_off)), 'r' + order.lower())
#q0=Quaternions.from_euler(rots, order=order.lower())
#q_off=Quaternions.from_euler(offset, order=order.lower())
#q2=((-q_off)*q0).euler(order=order.lower())
if use_deg:
q2 = np.rad2deg(q2)
return q2
def offsets(offset, rots, order='XYZ',use_deg=False):
if use_deg:
offset = np.deg2rad(offset)
rots = np.deg2rad(rots)
q0 = t3d.euler.euler2quat(rots[0], rots[1], rots[2], 'r' + order.lower())
q_off = t3d.euler.euler2quat(offset[0], offset[1], offset[2], 'r' + order.lower())
q2=t3d.euler.quat2euler(t3d.quaternions.qmult(q0,q_off), 'r' + order.lower())
#q0=Quaternions.from_euler(rots, order=order.lower())
#q_off=Quaternions.from_euler(offset, order=order.lower())
#q2=(q_off*q0).euler(order=order.lower())
if use_deg:
q2 = np.rad2deg(q2)
return q2
def euler2expmap2(rots, order='XYZ',use_deg=False):
if use_deg:
rots = np.deg2rad(rots)
#print("rot:" + str(rot))
quats=Quaternions.from_euler(rots, order=order.lower())
theta, vec = quats.angle_axis()
return unroll(vec*np.tile(theta[:,None],(1,3)))
def euler2expmap(rot, order='XYZ',use_deg=False):
if use_deg:
rot = np.deg2rad(rot)
#print("rot:" + str(rot))
vec, theta = t3d.euler.euler2axangle(rot[0], rot[1], rot[2], 'r' + order.lower())
return vec*theta
def expmap2euler(rot, order='XYZ',use_deg=False):
theta = np.linalg.norm(rot)
if theta > 1.0e-10:
vector = rot / theta
else:
vector = np.array([1.,0.,0.])
theta=0.0
eul = t3d.euler.axangle2euler(vector, theta, 'r' + order.lower())
if use_deg:
return np.rad2deg(eul)
else:
return eul
def euler2vectors(rot, order='XYZ', use_deg=False):
if use_deg:
rot = np.deg2rad(rot)
# order = "r" + (order.lower())[::-1] # try both r and s
# rotation_matrix = np
gitextract_9cw0czva/
├── .gitignore
├── Dockerfile
├── LICENSE
├── README.md
├── data/
│ └── motorica_dance/
│ ├── audio18_features.txt
│ ├── audio29_features.txt
│ ├── ch0_spec_beatact_features.txt
│ ├── dance_styles.txt
│ ├── dance_test_files.txt
│ ├── dance_train_files.txt
│ ├── data_pipe.expmap_30fps.sav
│ ├── gen_files.txt
│ ├── kthstreet_gKR_sFM_cAll_d01_mKR_ch01_chargedcableupyour_001_00.audio29_30fps.pkl
│ ├── kthstreet_gKR_sFM_cAll_d01_mKR_ch01_chargedcableupyour_001_00.expmap_30fps.pkl
│ ├── kthstreet_gKR_sFM_cAll_d01_mKR_ch01_chargedcableupyour_001_01.audio29_30fps.pkl
│ ├── kthstreet_gKR_sFM_cAll_d01_mKR_ch01_chargedcableupyour_001_01.expmap_30fps.pkl
│ ├── kthstreet_gLH_sFM_cAll_d02_mLH_ch01_lala_001_00.audio29_30fps.pkl
│ ├── kthstreet_gLH_sFM_cAll_d02_mLH_ch01_lala_001_00.expmap_30fps.pkl
│ ├── kthstreet_gLH_sFM_cAll_d02_mLH_ch01_lala_001_01.audio29_30fps.pkl
│ ├── kthstreet_gLH_sFM_cAll_d02_mLH_ch01_lala_001_01.expmap_30fps.pkl
│ ├── kthstreet_gLO_sFM_cAll_d02_mLO_ch01_arethafranklinrocksteady_002_00.audio29_30fps.pkl
│ ├── kthstreet_gLO_sFM_cAll_d02_mLO_ch01_arethafranklinrocksteady_002_00.expmap_30fps.pkl
│ ├── kthstreet_gLO_sFM_cAll_d02_mLO_ch01_arethafranklinrocksteady_002_01.audio29_30fps.pkl
│ ├── kthstreet_gLO_sFM_cAll_d02_mLO_ch01_arethafranklinrocksteady_002_01.expmap_30fps.pkl
│ ├── kthstreet_gPO_sFM_cAll_d01_mPO_ch01_bombom_002_00.audio29_30fps.pkl
│ ├── kthstreet_gPO_sFM_cAll_d01_mPO_ch01_bombom_002_00.expmap_30fps.pkl
│ ├── kthstreet_gPO_sFM_cAll_d01_mPO_ch01_bombom_002_01.audio29_30fps.pkl
│ ├── kthstreet_gPO_sFM_cAll_d01_mPO_ch01_bombom_002_01.expmap_30fps.pkl
│ ├── kthstreet_gPO_sFM_cAll_d02_mPO_ch01_bombom_001_00.audio29_30fps.pkl
│ ├── kthstreet_gPO_sFM_cAll_d02_mPO_ch01_bombom_001_00.expmap_30fps.pkl
│ ├── kthstreet_gPO_sFM_cAll_d02_mPO_ch01_bombom_001_01.audio29_30fps.pkl
│ ├── kthstreet_gPO_sFM_cAll_d02_mPO_ch01_bombom_001_01.expmap_30fps.pkl
│ └── pose_features.expmap.txt
├── experiments/
│ ├── dance_LDA-U.sh
│ ├── dance_LDA.sh
│ └── dance_mix_experts.sh
├── hparams/
│ ├── diffusion_dance_LDA-U.yaml
│ └── diffusion_dance_LDA.yaml
├── models/
│ ├── BaseModel.py
│ ├── LightningModel.py
│ ├── nn.py
│ └── transformer/
│ ├── tisa_transformer.py
│ └── tisa_v2.py
├── pretrained_models/
│ ├── .gitkeep
│ └── README.md
├── pymo/
│ ├── Pivots.py
│ ├── Quaternions.py
│ ├── __init__.py
│ ├── data.py
│ ├── features.py
│ ├── parsers.py
│ ├── preprocessing.py
│ ├── rotation_tools.py
│ ├── viz_tools.py
│ └── writers.py
├── requirements.txt
├── run_docker.sh
├── synthesize.py
├── train.py
└── utils/
├── cut_wav.py
├── download_from_youtube.py
├── duplicate_features.sh
├── hparams.py
├── logging_mixin.py
└── motion_dataset.py
SYMBOL INDEX (345 symbols across 19 files)
FILE: models/BaseModel.py
class BaseModel (line 15) | class BaseModel(LoggingMixin, LightningModule):
method __init__ (line 16) | def __init__(self, conf: Optional[Union[Dict, Namespace]] = None, **kw...
method get_scalers (line 34) | def get_scalers(self):
method standardizeInput (line 38) | def standardizeInput(self, input_tensor):
method standardizeOutput (line 42) | def standardizeOutput(self, output_tensor):
method destandardizeInput (line 46) | def destandardizeInput(self, input_tensor):
method destandardizeOutput (line 50) | def destandardizeOutput(self, predictions):
method configure_optimizers (line 53) | def configure_optimizers(self):
method on_before_optimizer_step (line 96) | def on_before_optimizer_step(self, optimizer, optimizer_idx):
FILE: models/LightningModel.py
class LitLDA (line 16) | class LitLDA(BaseModel):
method __init__ (line 17) | def __init__(self, conf, **kwargs):
method get_input_dim (line 56) | def get_input_dim(self):
method get_style_dim (line 59) | def get_style_dim(self):
method get_pose_dim (line 62) | def get_pose_dim(self):
method diffusion (line 65) | def diffusion(self, poses, t):
method forward (line 73) | def forward(self, batch):
method training_step (line 93) | def training_step(self, batch, batch_idx):
method validation_step (line 99) | def validation_step(self, batch, batch_idx):
method validation_epoch_end (line 116) | def validation_epoch_end(self, outputs):
method synthesize_and_log (line 120) | def synthesize_and_log(self, batch, log_prefix):
method test_step (line 128) | def test_step(self, batch, batch_idx):
method synthesize (line 136) | def synthesize(self, ctrl, global_cond):
FILE: models/nn.py
class Conv1dLayer (line 9) | class Conv1dLayer(nn.Module):
method __init__ (line 10) | def __init__(self, in_channels, out_channels, kernel_size, padding=0, ...
method forward (line 15) | def forward(self, x):
function silu (line 18) | def silu(x):
class DiffusionEmbedding (line 21) | class DiffusionEmbedding(nn.Module):
method __init__ (line 22) | def __init__(self, max_steps, in_channels, hidden_channels):
method forward (line 30) | def forward(self, diffusion_step):
method _lerp_embedding (line 41) | def _lerp_embedding(self, t):
method _build_embedding (line 48) | def _build_embedding(self, max_steps):
class ResidualBlock (line 55) | class ResidualBlock(nn.Module):
method __init__ (line 56) | def __init__(self, residual_channels,
method forward (line 81) | def forward(self, x, diffusion_step, local_cond):
class LDA (line 97) | class LDA(nn.Module):
method __init__ (line 98) | def __init__(self,
method forward (line 127) | def forward(self, x, local_cond, global_cond, diffusion_step):
FILE: models/transformer/tisa_transformer.py
class TisaTransformer (line 9) | class TisaTransformer(nn.Module):
method __init__ (line 11) | def __init__(
method forward (line 38) | def forward(self, x):
class Norm (line 45) | class Norm(nn.Module):
method __init__ (line 46) | def __init__(self, d_model, eps = 1e-6):
method forward (line 54) | def forward(self, x):
class AttnBlock (line 59) | class AttnBlock(nn.Module):
method __init__ (line 60) | def __init__(self, d_model, num_heads, activation, norm, drop_prob, ti...
method forward (line 80) | def forward(self, x):
class T5LayerNorm (line 95) | class T5LayerNorm(nn.Module):
method __init__ (line 96) | def __init__(self, hidden_size, eps=1e-6):
method forward (line 104) | def forward(self, hidden_states):
class ConvLayer (line 116) | class ConvLayer(nn.Module):
method __init__ (line 117) | def __init__(self, d_model, d_ff=2048, activation="relu", dilation=1, ...
method forward (line 127) | def forward(self, x):
class FeedForward (line 132) | class FeedForward(nn.Module):
method __init__ (line 133) | def __init__(self, d_model, d_ff=2048, activation="RELU", dropout = 0....
method forward (line 143) | def forward(self, x):
class GatedAttn (line 148) | class GatedAttn(nn.Module):
method __init__ (line 163) | def __init__(self, d_model, num_heads=4, activation="RELU", seqlen=128...
method forward (line 189) | def forward(self, x):
method dot_product_attention (line 213) | def dot_product_attention(self, q, k, v):
method split_last_dim (line 241) | def split_last_dim(x, n):
method combine_last_two_dim (line 263) | def combine_last_two_dim(x):
FILE: models/transformer/tisa_v2.py
class TisaV2 (line 8) | class TisaV2(nn.Module):
method __init__ (line 9) | def __init__(self,
method create_relative_offsets (line 81) | def create_relative_offsets(self, seq_len):
method forward (line 90) | def forward(self, dim=-1, skip_apply_dropout=False):
method _init_weights (line 119) | def _init_weights(self):
FILE: pymo/Pivots.py
class Pivots (line 5) | class Pivots:
method __init__ (line 19) | def __init__(self, ps): self.ps = np.array(ps)
method __str__ (line 20) | def __str__(self): return "Pivots("+ str(self.ps) + ")"
method __repr__ (line 21) | def __repr__(self): return "Pivots("+ repr(self.ps) + ")"
method __add__ (line 23) | def __add__(self, other): return Pivots(np.arctan2(np.sin(self.ps + ot...
method __sub__ (line 24) | def __sub__(self, other): return Pivots(np.arctan2(np.sin(self.ps - ot...
method __mul__ (line 25) | def __mul__(self, other): return Pivots(self.ps * other.ps)
method __div__ (line 26) | def __div__(self, other): return Pivots(self.ps / other.ps)
method __mod__ (line 27) | def __mod__(self, other): return Pivots(self.ps % other.ps)
method __pow__ (line 28) | def __pow__(self, other): return Pivots(self.ps ** other.ps)
method __lt__ (line 30) | def __lt__(self, other): return self.ps < other.ps
method __le__ (line 31) | def __le__(self, other): return self.ps <= other.ps
method __eq__ (line 32) | def __eq__(self, other): return self.ps == other.ps
method __ne__ (line 33) | def __ne__(self, other): return self.ps != other.ps
method __ge__ (line 34) | def __ge__(self, other): return self.ps >= other.ps
method __gt__ (line 35) | def __gt__(self, other): return self.ps > other.ps
method __abs__ (line 37) | def __abs__(self): return Pivots(abs(self.ps))
method __neg__ (line 38) | def __neg__(self): return Pivots(-self.ps)
method __iter__ (line 40) | def __iter__(self): return iter(self.ps)
method __len__ (line 41) | def __len__(self): return len(self.ps)
method __getitem__ (line 43) | def __getitem__(self, k): return Pivots(self.ps[k])
method __setitem__ (line 44) | def __setitem__(self, k, v): self.ps[k] = v.ps
method _ellipsis (line 46) | def _ellipsis(self): return tuple(map(lambda x: slice(None), self.shape))
method quaternions (line 48) | def quaternions(self, plane='xz'):
method directions (line 55) | def directions(self, plane='xz'):
method normalized (line 61) | def normalized(self):
method interpolate (line 67) | def interpolate(self, ws):
method copy (line 71) | def copy(self):
method shape (line 75) | def shape(self):
method from_quaternions (line 79) | def from_quaternions(cls, qs, forward='z', plane='xz'):
method from_directions (line 85) | def from_directions(cls, ds, plane='xz'):
FILE: pymo/Quaternions.py
class Quaternions (line 3) | class Quaternions:
method __init__ (line 23) | def __init__(self, qs):
method __str__ (line 36) | def __str__(self): return "Quaternions("+ str(self.qs) + ")"
method __repr__ (line 37) | def __repr__(self): return "Quaternions("+ repr(self.qs) + ")"
method _broadcast (line 42) | def _broadcast(cls, sqs, oqs, scalar=False):
method __add__ (line 66) | def __add__(self, other): return self * other
method __sub__ (line 67) | def __sub__(self, other): return self / other
method __mul__ (line 71) | def __mul__(self, other):
method __div__ (line 120) | def __div__(self, other):
method __eq__ (line 135) | def __eq__(self, other): return self.qs == other.qs
method __ne__ (line 136) | def __ne__(self, other): return self.qs != other.qs
method __neg__ (line 138) | def __neg__(self):
method __abs__ (line 142) | def __abs__(self):
method __iter__ (line 150) | def __iter__(self): return iter(self.qs)
method __len__ (line 151) | def __len__(self): return len(self.qs)
method __getitem__ (line 153) | def __getitem__(self, k): return Quaternions(self.qs[k])
method __setitem__ (line 154) | def __setitem__(self, k, v): self.qs[k] = v.qs
method lengths (line 157) | def lengths(self):
method reals (line 161) | def reals(self):
method imaginaries (line 165) | def imaginaries(self):
method shape (line 169) | def shape(self): return self.qs.shape[:-1]
method repeat (line 171) | def repeat(self, n, **kwargs):
method normalized (line 174) | def normalized(self):
method log (line 177) | def log(self):
method constrained (line 184) | def constrained(self, axis):
method constrained_x (line 201) | def constrained_x(self): return self.constrained(np.array([1,0,0]))
method constrained_y (line 202) | def constrained_y(self): return self.constrained(np.array([0,1,0]))
method constrained_z (line 203) | def constrained_z(self): return self.constrained(np.array([0,0,1]))
method dot (line 205) | def dot(self, q): return np.sum(self.qs * q.qs, axis=-1)
method copy (line 207) | def copy(self): return Quaternions(np.copy(self.qs))
method reshape (line 209) | def reshape(self, s):
method interpolate (line 213) | def interpolate(self, ws):
method euler (line 216) | def euler(self, order='xyz'):
method average (line 276) | def average(self):
method angle_axis (line 290) | def angle_axis(self):
method transforms (line 302) | def transforms(self):
method ravel (line 327) | def ravel(self):
method id (line 331) | def id(cls, n):
method id_like (line 346) | def id_like(cls, a):
method exp (line 352) | def exp(cls, ws):
method slerp (line 367) | def slerp(cls, q0s, q1s, a):
method between (line 396) | def between(cls, v0s, v1s):
method from_angle_axis (line 402) | def from_angle_axis(cls, angles, axis):
method from_euler (line 409) | def from_euler(cls, es, order='xyz', world=False):
method from_transforms (line 424) | def from_transforms(cls, ts):
FILE: pymo/data.py
class Joint (line 3) | class Joint():
method __init__ (line 4) | def __init__(self, name, parent=None, children=None):
class MocapData (line 9) | class MocapData():
method __init__ (line 10) | def __init__(self):
method traverse (line 18) | def traverse(self, j=None):
method clone (line 26) | def clone(self):
method get_all_channels (line 38) | def get_all_channels(self):
method get_skeleton_tree (line 44) | def get_skeleton_tree(self):
method get_empty_channels (line 50) | def get_empty_channels(self):
method get_constant_channels (line 54) | def get_constant_channels(self):
FILE: pymo/features.py
function get_foot_contact_idxs (line 12) | def get_foot_contact_idxs(signal, t=0.02, min_dist=120):
function create_foot_contact_signal (line 19) | def create_foot_contact_signal(mocap_track, col_name, start=1, t=0.02, m...
function plot_foot_up_down (line 36) | def plot_foot_up_down(mocap_track, col_name, t=0.02, min_dist=120):
FILE: pymo/parsers.py
class BVHScanner (line 15) | class BVHScanner():
method __init__ (line 19) | def __init__(self):
method scan (line 49) | def scan(self, stuff):
class BVHParser (line 54) | class BVHParser():
method __init__ (line 60) | def __init__(self, filename=None):
method reset (line 63) | def reset(self):
method parse (line 77) | def parse(self, filename, start=0, stop=-1):
method _to_DataFrame (line 95) | def _to_DataFrame(self):
method _new_bone (line 107) | def _new_bone(self, parent, name):
method _push_bone_context (line 111) | def _push_bone_context(self,name):
method _get_bone_context (line 114) | def _get_bone_context(self):
method _pop_bone_context (line 117) | def _pop_bone_context(self):
method _read_offset (line 121) | def _read_offset(self, bvh, token_index):
method _read_channels (line 131) | def _read_channels(self, bvh, token_index):
method _parse_joint (line 148) | def _parse_joint(self, bvh, token_index):
method _parse_hierarchy (line 187) | def _parse_hierarchy(self, bvh):
method _parse_motion (line 217) | def _parse_motion(self, bvh, start, stop):
FILE: pymo/preprocessing.py
class MocapParameterizer (line 22) | class MocapParameterizer(BaseEstimator, TransformerMixin):
method __init__ (line 23) | def __init__(self, param_type = 'euler', ref_pose=None):
method fit (line 34) | def fit(self, X, y=None):
method transform (line 37) | def transform(self, X, y=None):
method inverse_transform (line 59) | def inverse_transform(self, X, copy=None):
method _to_quat (line 78) | def _to_quat(self, X):
method _quat_to_euler (line 126) | def _quat_to_euler(self, X):
method _to_pos (line 167) | def _to_pos(self, X):
method _expmap2rot (line 249) | def _expmap2rot(self, expmap):
method _expmap_to_pos (line 278) | def _expmap_to_pos(self, X):
method _to_expmap (line 348) | def _to_expmap(self, X):
method _expmap_to_euler (line 392) | def _expmap_to_euler(self, X):
method _to_expmap2 (line 431) | def _to_expmap2(self, X):
method _expmap_to_euler2 (line 510) | def _expmap_to_euler2(self, X):
method _euler_to_vectors (line 570) | def _euler_to_vectors(self, X):
method _vectors_to_euler (line 614) | def _vectors_to_euler(self, X):
class Mirror (line 652) | class Mirror(BaseEstimator, TransformerMixin):
method __init__ (line 653) | def __init__(self, axis="X", append=True):
method fit (line 661) | def fit(self, X, y=None):
method transform (line 664) | def transform(self, X, y=None):
method inverse_transform (line 745) | def inverse_transform(self, X, copy=None, start_pos=None):
class EulerReorder (line 748) | class EulerReorder(BaseEstimator, TransformerMixin):
method __init__ (line 749) | def __init__(self, new_order):
method fit (line 756) | def fit(self, X, y=None):
method transform (line 760) | def transform(self, X, y=None):
method inverse_transform (line 814) | def inverse_transform(self, X, copy=None, start_pos=None):
class JointSelector (line 817) | class JointSelector(BaseEstimator, TransformerMixin):
method __init__ (line 821) | def __init__(self, joints, include_root=False):
method fit (line 825) | def fit(self, X, y=None):
method transform (line 848) | def transform(self, X, y=None):
method inverse_transform (line 867) | def inverse_transform(self, X, copy=None):
class Numpyfier (line 884) | class Numpyfier(BaseEstimator, TransformerMixin):
method __init__ (line 889) | def __init__(self):
method fit (line 892) | def fit(self, X, y=None):
method transform (line 898) | def transform(self, X, y=None):
method inverse_transform (line 908) | def inverse_transform(self, X, copy=None):
class Slicer (line 924) | class Slicer(BaseEstimator, TransformerMixin):
method __init__ (line 928) | def __init__(self, window_size, overlap=0.5):
method fit (line 933) | def fit(self, X, y=None):
method transform (line 939) | def transform(self, X, y=None):
method inverse_transform (line 960) | def inverse_transform(self, X, copy=None):
class RootTransformer (line 977) | class RootTransformer(BaseEstimator, TransformerMixin):
method __init__ (line 978) | def __init__(self, method, hips_axis_order="XYZ", position_smoothing=0...
method fit (line 998) | def fit(self, X, y=None):
method transform (line 1001) | def transform(self, X, y=None):
method inverse_transform (line 1291) | def inverse_transform(self, X, copy=None, start_pos=None):
class RootCentricPositionNormalizer (line 1528) | class RootCentricPositionNormalizer(BaseEstimator, TransformerMixin):
method __init__ (line 1529) | def __init__(self):
method fit (line 1532) | def fit(self, X, y=None):
method transform (line 1535) | def transform(self, X, y=None):
method inverse_transform (line 1570) | def inverse_transform(self, X, copy=None):
class Flattener (line 1598) | class Flattener(BaseEstimator, TransformerMixin):
method __init__ (line 1599) | def __init__(self):
method fit (line 1602) | def fit(self, X, y=None):
method transform (line 1605) | def transform(self, X, y=None):
class ConstantsRemover (line 1608) | class ConstantsRemover(BaseEstimator, TransformerMixin):
method __init__ (line 1613) | def __init__(self, eps = 1e-6):
method fit (line 1617) | def fit(self, X, y=None):
method transform (line 1624) | def transform(self, X, y=None):
method inverse_transform (line 1640) | def inverse_transform(self, X, copy=None):
class ListStandardScaler (line 1652) | class ListStandardScaler(BaseEstimator, TransformerMixin):
method __init__ (line 1653) | def __init__(self, is_DataFrame=False):
method fit (line 1656) | def fit(self, X, y=None):
method transform (line 1667) | def transform(self, X, y=None):
method inverse_transform (line 1684) | def inverse_transform(self, X, copy=None):
class ListMinMaxScaler (line 1702) | class ListMinMaxScaler(BaseEstimator, TransformerMixin):
method __init__ (line 1703) | def __init__(self, is_DataFrame=False):
method fit (line 1706) | def fit(self, X, y=None):
method transform (line 1717) | def transform(self, X, y=None):
method inverse_transform (line 1734) | def inverse_transform(self, X, copy=None):
class Resampler (line 1752) | class Resampler(BaseEstimator, TransformerMixin):
method __init__ (line 1753) | def __init__(self, fps, method='cubic'):
method fit (line 1762) | def fit(self, X, y=None):
method resample_dataframe (line 1767) | def resample_dataframe(self, df, frametime, method='cubic'):
method resample_df (line 1784) | def resample_df(self, df, new_frametime, old_frametime, mode='cubic'):
method transform (line 1821) | def transform(self, X, y=None):
method inverse_transform (line 1836) | def inverse_transform(self, X, copy=None):
class DownSampler (line 1851) | class DownSampler(BaseEstimator, TransformerMixin):
method __init__ (line 1852) | def __init__(self, tgt_fps, keep_all=False):
method fit (line 1857) | def fit(self, X, y=None):
method transform (line 1861) | def transform(self, X, y=None):
method inverse_transform (line 1887) | def inverse_transform(self, X, copy=None):
class ReverseTime (line 1890) | class ReverseTime(BaseEstimator, TransformerMixin):
method __init__ (line 1891) | def __init__(self, append=True):
method fit (line 1895) | def fit(self, X, y=None):
method transform (line 1899) | def transform(self, X, y=None):
method inverse_transform (line 1913) | def inverse_transform(self, X, copy=None):
class ListFeatureUnion (line 1916) | class ListFeatureUnion(BaseEstimator, TransformerMixin):
method __init__ (line 1917) | def __init__(self, processors):
method fit (line 1920) | def fit(self, X, y=None):
method transform (line 1933) | def transform(self, X, y=None):
method inverse_transform (line 1953) | def inverse_transform(self, X, y=None):
class RollingStatsCalculator (line 1956) | class RollingStatsCalculator(BaseEstimator, TransformerMixin):
method __init__ (line 1960) | def __init__(self, win):
method fit (line 1963) | def fit(self, X, y=None):
method transform (line 1967) | def transform(self, X, y=None):
method inverse_transform (line 1989) | def inverse_transform(self, X, copy=None):
class FeatureCounter (line 1992) | class FeatureCounter(BaseEstimator, TransformerMixin):
method __init__ (line 1993) | def __init__(self):
method fit (line 1996) | def fit(self, X, y=None):
method transform (line 2001) | def transform(self, X, y=None):
method inverse_transform (line 2004) | def inverse_transform(self, X, copy=None):
class TemplateTransform (line 2013) | class TemplateTransform(BaseEstimator, TransformerMixin):
method __init__ (line 2014) | def __init__(self):
method fit (line 2017) | def fit(self, X, y=None):
method transform (line 2020) | def transform(self, X, y=None):
FILE: pymo/rotation_tools.py
function deg2rad (line 15) | def deg2rad(x):
function rad2deg (line 19) | def rad2deg(x):
function unroll (line 22) | def unroll(rots):
function unroll_1 (line 32) | def unroll_1(rots):
function unroll_2 (line 58) | def unroll_2(rots):
function euler_reorder2 (line 91) | def euler_reorder2(rots, order='XYZ', new_order='XYZ',use_deg=False):
function euler_reorder (line 106) | def euler_reorder(rot, order='XYZ', new_order='XYZ',use_deg=False):
function offsets_inv (line 124) | def offsets_inv(offset, rots, order='XYZ',use_deg=False):
function offsets (line 141) | def offsets(offset, rots, order='XYZ',use_deg=False):
function euler2expmap2 (line 158) | def euler2expmap2(rots, order='XYZ',use_deg=False):
function euler2expmap (line 166) | def euler2expmap(rot, order='XYZ',use_deg=False):
function expmap2euler (line 173) | def expmap2euler(rot, order='XYZ',use_deg=False):
function euler2vectors (line 187) | def euler2vectors(rot, order='XYZ', use_deg=False):
function vectors2euler (line 204) | def vectors2euler(axises, order='XYZ', use_deg=False):
class Rotation (line 226) | class Rotation():
method __init__ (line 227) | def __init__(self,rot, param_type, **params):
method _from_euler (line 234) | def _from_euler(self, alpha, beta, gamma, params):
method _from_expmap (line 283) | def _from_expmap(self, alpha, beta, gamma, params):
method get_euler_axis (line 309) | def get_euler_axis(self):
method to_expmap (line 316) | def to_expmap(self):
method to_euler (line 324) | def to_euler(self, use_deg=False, order='xyz'):
method to_quat (line 363) | def to_quat(self):
method __str__ (line 367) | def __str__(self):
FILE: pymo/viz_tools.py
function save_fig (line 10) | def save_fig(fig_id, tight_layout=True):
function draw_stickfigure (line 16) | def draw_stickfigure(mocap_track, frame, data=None, joints=None, draw_na...
function draw_stickfigure3d (line 53) | def draw_stickfigure3d(mocap_track, frame, data=None, joints=None, draw_...
function sketch_move (line 102) | def sketch_move(mocap_track, data=None, ax=None, figsize=(16,8)):
function render_mp4 (line 127) | def render_mp4(mocap_track, filename, data=None, ax=None, axis_scale=50,...
function viz_cnn_filter (line 200) | def viz_cnn_filter(feature_to_viz, mocap_track, data, gap=25):
function print_skel (line 230) | def print_skel(X):
FILE: pymo/writers.py
class BVHWriter (line 4) | class BVHWriter():
method __init__ (line 5) | def __init__(self):
method write (line 8) | def write(self, X, ofile, framerate=-1, start=0, stop=-1):
method _printJoint (line 36) | def _printJoint(self, X, joint, tab, ofile):
FILE: synthesize.py
function sample_mixmodels (line 11) | def sample_mixmodels(models, batches, guidance_factors):
function do_synthesize (line 71) | def do_synthesize(models, l_conds, g_conds, file_name, postfix, trim, de...
function nans2zeros (line 86) | def nans2zeros(x):
function get_style_vector (line 93) | def get_style_vector(styles_file, style_token, nbatch, nframes):
function get_cond (line 98) | def get_cond(model, data_dir, input_file, style_token, length):
function arg2tokens (line 124) | def arg2tokens(arg, delim=","):
function arg2tokens_f (line 127) | def arg2tokens_f(arg, delim=","):
FILE: train.py
function data_loader (line 19) | def data_loader(dataset_root, file_name, data_hparams, batch_size, num_w...
function dataloaders (line 37) | def dataloaders(dataset_root, data_hparams, batch_size, num_workers):
FILE: utils/hparams.py
function get_hparams (line 14) | def get_hparams():
FILE: utils/logging_mixin.py
class LoggingMixin (line 18) | class LoggingMixin:
method log_results (line 20) | def log_results(self, pred_clips, file_name, log_prefix, logdir=None, ...
method feats_to_bvh (line 44) | def feats_to_bvh(self, pred_clips):
method write_bvh (line 54) | def write_bvh(self, bvh_data, log_dir="", name_prefix=""):
method bvh_to_pos (line 66) | def bvh_to_pos(self, bvh_data):
method render_video (line 70) | def render_video(self, pos_data, log_dir="", name_prefix=""):
method log_jerk (line 82) | def log_jerk(self, x, log_prefix):
FILE: utils/motion_dataset.py
function concat_dataframes (line 17) | def concat_dataframes(x,y):
function nans2zeros (line 28) | def nans2zeros(x):
function dataframe_nansinf2zeros (line 35) | def dataframe_nansinf2zeros(df):
function align_start (line 40) | def align_start(x,y):
function parse_token (line 46) | def parse_token(f_name, inds):
function styles2onehot (line 58) | def styles2onehot(all_styles, style_token):
function files_to_list (line 67) | def files_to_list(filename):
function resample_data (line 77) | def resample_data(data, nframes_new, has_root_motion, mode='linear'):
class MotionDataset (line 95) | class MotionDataset(torch.utils.data.Dataset):
method __init__ (line 96) | def __init__(self, data_root, datafiles_file, data_hparams=None):
method assert_not_const (line 176) | def assert_not_const(self, data):
method fit_scalers (line 180) | def fit_scalers(self):
method standardize (line 199) | def standardize(self, scalers):
method timestretch (line 203) | def timestretch(self, data, segment_length, factor, has_root_motion=Fa...
method __getitem__ (line 214) | def __getitem__(self, index):
method __len__ (line 239) | def __len__(self):
Condensed preview — 65 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (254K chars).
[
{
"path": ".gitignore",
"chars": 24,
"preview": "results/*\n**/__pycache__"
},
{
"path": "Dockerfile",
"chars": 239,
"preview": "FROM nvcr.io/nvidia/pytorch:22.10-py3\nENV DEBIAN_FRONTEND=noninteractive\n\nWORKDIR /workspace\n\nRUN apt-get update\nRUN apt"
},
{
"path": "LICENSE",
"chars": 1169,
"preview": "License and Copyright Information\n---------------------------------\nThe contents of this repository may not be used for "
},
{
"path": "README.md",
"chars": 3286,
"preview": "# Listen, Denoise, Action!\nThis repository provides code and models for the paper [Listen, denoise, action! Audio-driven"
},
{
"path": "data/motorica_dance/audio18_features.txt",
"chars": 170,
"preview": "MFCC_0\nMFCC_1\nMFCC_2\nChroma_0\nChroma_1\nChroma_2\nChroma_3\nChroma_4\nChroma_5\nChroma_6\nChroma_7\nChroma_8\nChroma_9\nChroma_10"
},
{
"path": "data/motorica_dance/audio29_features.txt",
"chars": 243,
"preview": "MFCC_0\nMFCC_1\nMFCC_2\nMFCC_3\nMFCC_4\nMFCC_5\nMFCC_6\nMFCC_7\nMFCC_8\nMFCC_9\nMFCC_10\nMFCC_11\nMFCC_12\nMFCC_13\nMFCC_14\nMFCC_15\nMF"
},
{
"path": "data/motorica_dance/ch0_spec_beatact_features.txt",
"chars": 41,
"preview": "Chroma_0\nSpectralflux_0\nBeatactivation_0\n"
},
{
"path": "data/motorica_dance/dance_styles.txt",
"chars": 32,
"preview": "gCA\ngCH\ngJZ\ngKR\ngLH\ngLO\ngPO\ngTP\n"
},
{
"path": "data/motorica_dance/dance_test_files.txt",
"chars": 1088,
"preview": "kthjazz_gCH_sFM_cAll_d02_mCH_ch01_whitemanpaulandhisorchestraloisiana_006_00\nkthjazz_gCH_sFM_cAll_d02_mCH_ch01_whitemanp"
},
{
"path": "data/motorica_dance/dance_train_files.txt",
"chars": 23020,
"preview": "kthjazz_gCH_sFM_cAll_d02_mCH_ch01_beatlestreetwashboardbandfortyandtight_003_00\nkthjazz_gCH_sFM_cAll_d02_mCH_ch01_beatle"
},
{
"path": "data/motorica_dance/gen_files.txt",
"chars": 410,
"preview": "kthjazz_gCH_sFM_cAll_d02_mCH_ch01_whitemanpaulandhisorchestraloisiana_006_00\nkthjazz_gJZ_sFM_cAll_d02_mJZ_ch01_bennygood"
},
{
"path": "data/motorica_dance/pose_features.expmap.txt",
"chars": 920,
"preview": "RightFoot_alpha\nRightFoot_beta\nRightFoot_gamma\nRightLeg_alpha\nRightLeg_beta\nRightLeg_gamma\nRightUpLeg_alpha\nRightUpLeg_b"
},
{
"path": "experiments/dance_LDA-U.sh",
"chars": 1408,
"preview": "#!/bin/bash\n\ncheckpoint=pretrained_models/dance_LDA-U.ckpt\ndest_dir=results/generated/dance_LDA-U\n\nif [ ! -d \"${dest_dir"
},
{
"path": "experiments/dance_LDA.sh",
"chars": 1512,
"preview": "#!/bin/bash\n\ncheckpoint=pretrained_models/dance_LDA.ckpt\ndest_dir=results/generated/dance_LDA\n\nif [ ! -d \"${dest_dir}\" ]"
},
{
"path": "experiments/dance_mix_experts.sh",
"chars": 2252,
"preview": "#!/bin/bash\n\n### Mixing N models/conditionings to generate a mix of styles. 1. Model A trained w.o. style conditioning 2"
},
{
"path": "hparams/diffusion_dance_LDA-U.yaml",
"chars": 1768,
"preview": "Data:\n segment_length: 150\n style_index: [1]\n trim_edges: 300\n traindata_filename: dance_train_files_kth.txt\n testd"
},
{
"path": "hparams/diffusion_dance_LDA.yaml",
"chars": 1781,
"preview": "Data:\n segment_length: 150\n style_index: [1]\n trim_edges: 300\n styles_file: dance_styles_kth.txt\n traindata_filenam"
},
{
"path": "models/BaseModel.py",
"chars": 4012,
"preview": "# Copyright 2023 Motorica AB, Inc. All Rights Reserved.\n\nimport os\n\nimport torch\nimport numpy as np\nfrom torch.optim imp"
},
{
"path": "models/LightningModel.py",
"chars": 6947,
"preview": "# Copyright 2023 Motorica AB, Inc. All Rights Reserved.\n\nimport os\nimport sys\n\nimport numpy as np\nimport torch\nfrom torc"
},
{
"path": "models/nn.py",
"chars": 5337,
"preview": "# Copyright 2023 Motorica AB, Inc. All Rights Reserved.\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional a"
},
{
"path": "models/transformer/tisa_transformer.py",
"chars": 9870,
"preview": "# Copyright 2023 Motorica AB, Inc. All Rights Reserved.\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional a"
},
{
"path": "models/transformer/tisa_v2.py",
"chars": 4567,
"preview": "# Copyright 2023 Ulme Wennberg, Inc. All Rights Reserved.\n\nimport torch\nfrom torch import nn\nfrom typing import List\nimp"
},
{
"path": "pretrained_models/.gitkeep",
"chars": 0,
"preview": ""
},
{
"path": "pretrained_models/README.md",
"chars": 117,
"preview": "Please [download our pretrained dance models here](https://zenodo.org/record/8156769) and place them in this folder.\n"
},
{
"path": "pymo/Pivots.py",
"chars": 3287,
"preview": "import numpy as np\n\nfrom pymo.Quaternions import Quaternions\n\nclass Pivots: \n \"\"\"\n Pivots is an ndarray of angu"
},
{
"path": "pymo/Quaternions.py",
"chars": 17160,
"preview": "import numpy as np\n\nclass Quaternions:\n \"\"\"\n Quaternions is a wrapper around a numpy ndarray\n that allows it to"
},
{
"path": "pymo/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "pymo/data.py",
"chars": 1682,
"preview": "import numpy as np\n\nclass Joint():\n def __init__(self, name, parent=None, children=None):\n self.name = name\n "
},
{
"path": "pymo/features.py",
"chars": 1255,
"preview": "'''\nA set of mocap feature extraction functions\n\nCreated by Omid Alemi | Nov 17 2017\n\n'''\nimport numpy as np\nimport pand"
},
{
"path": "pymo/parsers.py",
"chars": 8954,
"preview": "'''\nBVH Parser Class\n\nBy Omid Alemi\nCreated: June 12, 2017\n\nBased on: https://gist.github.com/johnfredcee/2007503\n\n'''\ni"
},
{
"path": "pymo/preprocessing.py",
"chars": 87413,
"preview": "'''\nPreprocessing Tranformers Based on sci-kit's API\n\nBy Omid Alemi\nCreated on June 12, 2017\n'''\nimport copy\nimport pand"
},
{
"path": "pymo/rotation_tools.py",
"chars": 11540,
"preview": "'''\nTools for Manipulating and Converting 3D Rotations\n\nBy Omid Alemi\nCreated: June 12, 2017\n\nAdapted from that matlab f"
},
{
"path": "pymo/viz_tools.py",
"chars": 10868,
"preview": "import pandas as pd\nimport numpy as np\nimport matplotlib.animation as animation\nimport matplotlib.colors as colors\nimpor"
},
{
"path": "pymo/writers.py",
"chars": 2790,
"preview": "import numpy as np\nimport pandas as pd\n\nclass BVHWriter():\n def __init__(self):\n pass\n \n def write(self,"
},
{
"path": "requirements.txt",
"chars": 247,
"preview": "numpy\nscipy\nlibrosa\ntorchmetrics\npytorch-lightning==1.8.3\npandas\nmatplotlib\nscikit-learn==0.24.2\nmpi4py\njoblib\nsmplx\ntra"
},
{
"path": "run_docker.sh",
"chars": 178,
"preview": "docker run -it --rm --gpus '\"device=0,1,2,3,4,5,6,7\"' -v $PWD:/workspace/dockers --ipc=host -v=$HOME/data:/workspace/doc"
},
{
"path": "synthesize.py",
"chars": 8226,
"preview": "# Copyright 2023 Motorica AB, Inc. All Rights Reserved.\nfrom os.path import join\nimport os, sys, getopt\nimport torch\nimp"
},
{
"path": "train.py",
"chars": 3206,
"preview": "# Copyright 2023 Motorica AB, Inc. All Rights Reserved.\n\nimport os\nimport sys\n\nimport numpy as np\nfrom utils.motion_data"
},
{
"path": "utils/cut_wav.py",
"chars": 945,
"preview": "# Copyright 2023 Motorica AB, Inc. All Rights Reserved.\n\nimport numpy as np\nimport scipy.io.wavfile as wav\n\nimport os\nim"
},
{
"path": "utils/download_from_youtube.py",
"chars": 2567,
"preview": "# import pafy\n\n# #URL of the YouTube video\n# url = \"https://www.youtube.com/watch?v=dQw4w9WgXcQ\"\n\n# #Create a pafy objec"
},
{
"path": "utils/duplicate_features.sh",
"chars": 487,
"preview": "#!/bin/bash\n#./duplicate_features.sh data/GENEA/processed_60fps audio14_60fps\naugmentation=mirrored\nfind $1 -name \"*.${2"
},
{
"path": "utils/hparams.py",
"chars": 1149,
"preview": "import os\nimport numpy as np\n\nfrom argparse import ArgumentParser, Namespace\nfrom jsmin import jsmin\nimport json\nimport "
},
{
"path": "utils/logging_mixin.py",
"chars": 3168,
"preview": "# Copyright 2023 Motorica AB, Inc. All Rights Reserved.\n\nimport io\nimport json\nfrom pathlib import Path\nimport joblib as"
},
{
"path": "utils/motion_dataset.py",
"chars": 9262,
"preview": "# Copyright 2023 Motorica AB, Inc. All Rights Reserved.\n\nimport os\nimport random\nimport argparse\nimport json\nimport torc"
}
]
// ... and 21 more files (download for full content)
About this extraction
This page contains the full source code of the simonalexanderson/ListenDenoiseAction GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 65 files (238.9 KB), approximately 71.0k tokens, and a symbol index with 345 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.