![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
"
]
},
{
"cell_type": "markdown",
"source": [
"# **Demo of Controllable Music Transformer**\n",
"\n",
"We provide a colab notebook for running inference with CMT. You can upload a video and generate a background music using this notebook."
],
"metadata": {
"id": "Qx-JUsBYpv3X"
}
},
{
"cell_type": "markdown",
"source": [
"# 1. Preparation"
],
"metadata": {
"id": "iGsKCYiR8ZBy"
}
},
{
"cell_type": "markdown",
"source": [
"Clone the repo"
],
"metadata": {
"id": "t34LcwtQGq7_"
}
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "YMTYugKn6NNp"
},
"outputs": [],
"source": [
"import os\n",
"from google.colab import files\n",
"import json\n",
"\n",
"os.chdir('/content')\n",
"!git clone https://github.com/wzk1015/video-bgm-generation\n",
"os.chdir('/content/video-bgm-generation')"
]
},
{
"cell_type": "markdown",
"source": [
"Download checkpoint and soundfont\n",
"\n"
],
"metadata": {
"id": "X9ZN2EbvG9J4"
}
},
{
"cell_type": "code",
"source": [
"!gsutil -m cp gs://cmt/loss_8_params.pt /content/video-bgm-generation/exp/\n",
"!gsutil -m cp gs://magentadata/soundfonts/SGM-v2.01-Sal-Guit-Bass-V1.3.sf2 /content/video-bgm-generation/"
],
"metadata": {
"id": "c7XER9vH8mfb"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Install dependencies"
],
"metadata": {
"id": "b4RRCIybl7_I"
}
},
{
"cell_type": "code",
"source": [
"!apt-get update && apt-get install libfluidsynth1 build-essential libasound2-dev libjack-dev fluidsynth"
],
"metadata": {
"id": "efg5Ya8cJL5o"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"!pip install --upgrade pip\n",
"# this may take ~15 minutes\n",
"!pip install pytorch-fast-transformers==0.3.0\n",
"# Note: Version of pytorch-fast-transformers is tricky - depends on your randomly assigned colab GPU, it could be 0.3.0 or 0.4.0 or others.\n",
"# Incorrect fast-transformers version could lead to Errors or generating awful results for unknown reasons,\n",
"# so you should try different versions, or refer to https://github.com/idiap/fast-transformers\n",
"\n",
"!pip install -r py3_requirements.txt\n",
"os.chdir(\"/content/video-bgm-generation/src/video2npz/visbeat3/\")\n",
"!python setup.py install"
],
"metadata": {
"id": "SkRSrynzSrA-"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"# 2. Process input video"
],
"metadata": {
"id": "ygRjsNf5F0FT"
}
},
{
"cell_type": "markdown",
"source": [
"Upload your video\n",
"\n",
"It is recommended to use videos **less than 2 minutes**, otherwise it gets really slow"
],
"metadata": {
"id": "QMS5SlFOrVv-"
}
},
{
"cell_type": "code",
"source": [
"os.chdir(\"/content/video-bgm-generation/\")\n",
"uploaded = files.upload()\n",
"assert len(uploaded) == 1, \"upload one video file only\"\n",
"filename = list(uploaded.keys())[0]\n",
"os.system(f'mv {filename} videos/test_raw.mp4')"
],
"metadata": {
"id": "gczMG7TyQYCC"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Convert to 360p to speed up extracting optical flow and visbeats"
],
"metadata": {
"id": "dR5dCMo5qfk-"
}
},
{
"cell_type": "code",
"source": [
"os.chdir(\"/content/video-bgm-generation/videos/\")\n",
"!rm test.mp4\n",
"!ffmpeg -i test_raw.mp4 -strict -2 -vf scale=-1:360 test.mp4"
],
"metadata": {
"id": "RzRrVu9RqTZ6"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Extracting optical flow and visbeats, convert video into npz file"
],
"metadata": {
"id": "sgm_TOq7QYXn"
}
},
{
"cell_type": "code",
"source": [
"os.chdir(\"/content/video-bgm-generation/src/video2npz/\")\n",
"!rm -r VisBeatAssets/ fig/ flow/ image/ optical_flow/\n",
"!bash video2npz.sh ../../videos/test.mp4\n",
"# extracting optical flow and visbeats may be slow"
],
"metadata": {
"id": "y_l8VDLFFE-c"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"# 3. Run the model to generate background music"
],
"metadata": {
"id": "-JASD-zxJZJt"
}
},
{
"cell_type": "markdown",
"source": [
"Run inference to generate MIDI (.mid) output"
],
"metadata": {
"id": "TpZnfeIsHhyM"
}
},
{
"cell_type": "code",
"source": [
"os.chdir(\"/content/video-bgm-generation/src/\")\n",
"!python gen_midi_conditional.py -f \"../inference/test.npz\" -c \"../exp/loss_8_params.pt\" -n 1"
],
"metadata": {
"id": "Xxus1H-XGHXj"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Convert midi into audio: use **GarageBand (recommended)** or midi2audio\n",
"\n",
"Remember to **set tempo to the value of tempo in video2npz/metadata.json**"
],
"metadata": {
"id": "pR3rUJWIJnVp"
}
},
{
"cell_type": "code",
"source": [
"os.chdir(\"/content/video-bgm-generation/src/\")\n",
"files.download('../inference/test.npz_0.mid')\n",
"\n",
"with open(\"video2npz/metadata.json\") as f:\n",
" tempo = json.load(f)['tempo']\n",
" print(\"tempo:\", tempo)"
],
"metadata": {
"id": "lKR7qWinMUFM"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Generate audio with midi2audio\n",
"\n",
"Instead of running this cell, we recommend using GarageBand or other softwares, since their soundfonts are better. But this also works fine"
],
"metadata": {
"id": "GzEb08C4ovjD"
}
},
{
"cell_type": "code",
"source": [
"import note_seq\n",
"from pretty_midi import PrettyMIDI\n",
"import midi2audio\n",
"import numpy as np\n",
"import io\n",
"import scipy\n",
"\n",
"SAMPLE_RATE = 16000\n",
"SF2_PATH = '/content/video-bgm-generation/SGM-v2.01-Sal-Guit-Bass-V1.3.sf2'\n",
"os.chdir(\"/content/video-bgm-generation/inference/\")\n",
"\n",
"input_mid = 'test.npz_0.mid'\n",
"midi_obj = PrettyMIDI(input_mid)\n",
"# convert tempo\n",
"midi_length = midi_obj.get_end_time()\n",
"midi_obj.adjust_times([0, midi_length], [0, midi_length*120/tempo])\n",
"processed_mid = input_mid[:-4] + \"_processed.mid\"\n",
"midi_obj.write(processed_mid)\n",
"print(\"converting into mp3\")\n",
"fs = midi2audio.FluidSynth(SF2_PATH, sample_rate=SAMPLE_RATE)\n",
"fs.midi_to_audio(processed_mid, \"music.mp3\")\n",
"\n",
"print(\"playing music\")\n",
"ns = note_seq.midi_io.midi_to_note_sequence(midi_obj)\n",
"note_seq.play_sequence(ns, synth=note_seq.fluidsynth, sample_rate=SAMPLE_RATE, sf2_path=SF2_PATH)\n",
"note_seq.plot_sequence(ns)\n",
" "
],
"metadata": {
"id": "fZHzA0UtKDWa"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Combine original video and audio into video with BGM\n",
"\n",
"Generate/upload the audio file under `inference`, name it as `music.mp3`, and run this to combine video and music"
],
"metadata": {
"id": "KKObBCKBKlU1"
}
},
{
"cell_type": "code",
"source": [
"os.chdir(\"/content/video-bgm-generation/inference/\")\n",
"!rm output.mp4\n",
"!ffmpeg -i ../videos/test_raw.mp4 -i music.mp3 -c:v copy -c:a aac -strict experimental -map 0:v:0 -map 1:a:0 output.mp4\n",
"files.download('output.mp4')"
],
"metadata": {
"id": "SqNLXFzmLPjP"
},
"execution_count": null,
"outputs": []
}
]
}
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2021 Shangzhe Di
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: README.md
================================================
# Controllable Music Transformer
Official code for our paper *Video Background Music Generation with Controllable Music Transformer* (ACM MM 2021 **Best Paper Award**)
[[Paper]](https://arxiv.org/abs/2111.08380) [[Project Page]](https://wzk1015.github.io/cmt/) [[Colab Demo]](https://colab.research.google.com/github/wzk1015/video-bgm-generation/blob/main/CMT.ipynb)
## News
[2025.3] **Take a look at our survey on vision-to-music generation, published in ISMIR 2025 ([Paper](https://arxiv.org/abs/2503.21254), [Repo](https://github.com/wzk1015/Awesome-Vision-to-Music-Generation))**, with the latest methods, datasets and evaluation in video-to-music and image-to-music generation.
[2024.12] 🚀🚀 **Check out our new [paper](https://github.com/wbs2788/VMB/tree/main) for multimodal music generation!** We propose a novel multimodal music generation approach with explicit text and music bridges for video-to-music, image-to-music, text-to-music, and controllable music generation tasks.
[2023.9] **Check out our new [ICCV 2023 paper](https://arxiv.org/abs/2211.11248) for video background music generation.** We provide a video and symbolic music dataset with rich annotations, an objective metric for video-music correspondence, and a benchmark model that utilizes music priors of chords, melody, and accompaniment along with video-music relations of semantic, color, and motion features.
[2022.5] **We provide a [colab notebook](https://colab.research.google.com/github/wzk1015/video-bgm-generation/blob/main/CMT.ipynb) for demo!** You can run inference code and generate background music for your input video.
[2021.10] 🏆 We won the **ACM MM 2021 [Best Paper Award](https://www.wzk.plus/award_imgs/mm.jpg)**!
## Introduction
We address the unexplored task – *video background music generation*. We first establish three rhythmic relations between video and background music. We then propose a **C**ontrollable **M**usic **T**ransformer (CMT) to achieve local and global control of the music generation process. Our proposed method does not require paired video and music data for training while generating melodious and compatible music with the given video.
## Directory Structure
* `src/`: code of the whole pipeline
* `train.py`: training script, take an npz file as input music data to train the model
* `model.py`: code of the model
* `gen_midi_conditional.py`: inference script, take an npz file (represents a video) as input to generate several songs
* `midi2mp3.py`: script of converting midi into mp3
* `src/video2npz/`: convert video into npz by extracting motion saliency and motion speed
* `dataset/`: processed dataset for training, in the format of npz
* `logs/`: logs that automatically generated during training, can be used to track the training process
* `exp/`: checkpoints, named after val loss (e.g. `loss_8_params.pt`)
* `inference/`: processed video for inference (.npz), and generated music(.mid)
## Preparation
* Clone this repo
* Download the processed training data `lpd_5_prcem_mix_v8_10000.npz` from [HERE](https://drive.google.com/file/d/19f_DytIbEiSDCwz8FPpScrHqmnqNtVYT/view?usp=sharing) and put it under `dataset/`
* Download the pre-trained model `loss_8_params.pt` from [HERE](https://drive.google.com/file/d/1KvIRRm0KqTlEFDjAgs4fMtLRQBq0rBWy/view?usp=sharing) and put it under `exp/`
* Install `ffmpeg=3.2.4`
* Install Python3 dependencies `pip install -r py3_requirements.txt`
* Choose the correct version of `torch` and `pytorch-fast-transformers` based on your CUDA version (see [fast-transformers repo](https://github.com/idiap/fast-transformers) and [this issue](https://github.com/wzk1015/video-bgm-generation/issues/3))
* Install `visbeat3` package: `cd src/video2npz/visbeat3; python setup.py install`
* (Optional) If you want to convert midi into mp3 with midi2audio:
* Install fluidsynth following [this](https://github.com/FluidSynth/fluidsynth/wiki/Download)
* Download soundfont `SGM-v2.01-Sal-Guit-Bass-V1.3.sf2` from [HERE](https://drive.google.com/file/d/1zDg0P-0rCXDl_wX4zeLcKRNmOFobq6u8/view?usp=sharing) and put it directly under this folder (`video-bgm-generation`)
## Training
- A quick start by using the processed data `lpd_5_prcem_mix_v8_10000.npz` (1~2 days on 8x 1080Ti GPUs):
```shell
python train.py --name train_default -b 8 --gpus 0 1 2 3 4 5 6 7
```
* (Optional) If you want to reproduce the whole process:
1. Download the lpd-5-cleansed dataset from [HERE](https://drive.google.com/file/d/1AzLZ4fHrcek1rVNlOC3pzxsMaNSITZsG/view?usp=sharing) and put the extracted files under `dataset/lpd_5_cleansed/`
2. Go to `src/` and convert the pianoroll files (.npz) to midi files (~3 files / sec):
```shell
python pianoroll2midi.py --in_dir ../dataset/lpd_5_cleansed/ --out_dir ../dataset/lpd_5_cleansed_midi/
```
3. Convert midi files to .npz files with our proposed representation (~5 files / sec):
```shell
python midi2numpy_mix.py --midi_dir ../dataset/lpd_5_cleansed_midi/ --out_name data.npz
```
4. Train the model (1~2 days on 8x 1080Ti GPUs):
```shell
python train.py --name train_exp --train_data ../dataset/data.npz -b 8 --gpus 0 1 2 3 4 5 6 7
```
**Note:** If you want to train with another MIDI dataset, please ensure that each track belongs to one of the five instruments (Drums, Piano, Guitar, Bass, or Strings) and is named exactly with its instrument. Otherwise, you may have to change the track names in the MIDI files. You can check this with [Muspy](https://salu133445.github.io/muspy/):
```python
import muspy
midi = muspy.read_midi('xxx.mid')
print([track.name for track in midi.tracks]) # Should be like ['Drums', 'Guitar', 'Bass', 'Strings']
```
## Inference
Inference requires one GPU. You can try our [colab notebook](https://colab.research.google.com/github/wzk1015/video-bgm-generation/blob/main/CMT.ipynb) to run inference.
It is recommended to use videos *less than 2 minutes*, otherwise, it gets really slow
* Resize the video into 360p
```shell
ffmpeg -i xxx.mp4 -strict -2 -vf scale=-1:360 test.mp4
```
* Convert input video (MP4 format) into npz
```shell
cd src/video2npz
sh video2npz.sh ../../videos/test.mp4
```
* Run model to generate `.mid` :
```shell
python gen_midi_conditional.py -f "../inference/test.npz" -c "../exp/loss_8_params.pt" -n 5
# If using another training set, change `decoder_n_class` in `gen_midi_conditional` to the one in `train.py`
```
* Convert midi into audio
* Get tempo of the music:
* ```python
# metadata.json is generated when running `video2npz.sh`
with open("video2npz/metadata.json") as f:
tempo = json.load(f)['tempo']
print("tempo:", tempo)
```
* (A) Use GarageBand to convert midi into audio
* this is **recommended** since their soundfonts are better, and no need to install fluidsynth and soundfonts
* remember to set tempo
* (B) Use midi2audio
```shell
# Make sure you have installed fluidsynth and downloaded soundfont
python midi2mp3.py --input ../inference/get_0.mid --output ../inference/get_0.mp3
```
* Combine original video and audio into video with BGM:
````shell
ffmpeg -i test.mp4 -i get_0.mp3 -c:v copy -c:a aac -strict experimental -map 0:v:0 -map 1:a:0 output.mp4
# test.mp4: input video
# get_0.mp3: audio file generated in the previous step
# output.mp4: output video with BGM
````
## Matching Method
The matching method finds the five most matching music pieces from the music library for a given video.
```shell
python src/match.py inference/test.npz dataset/lpd_5_prcem_mix_v8_10000.npz
```
## Citation
```bibtex
@inproceedings{di2021video,
title={Video Background Music Generation with Controllable Music Transformer},
author={Di, Shangzhe and Jiang, Zeren and Liu, Si and Wang, Zhaokai and Zhu, Leyan and He, Zexin and Liu, Hongming and Yan, Shuicheng},
booktitle={ACM Multimedia},
year={2021}
}
@inproceedings{zhuo2023video,
title={Video background music generation: Dataset, method and evaluation},
author={Zhuo, Le and Wang, Zhaokai and Wang, Baisen and Liao, Yue and Bao, Chenxi and Peng, Stanley and Han, Songhao and Zhang, Aixi and Fang, Fei and Liu, Si},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
pages={15637--15647},
year={2023}
}
@article{vmb,
title={Multimodal Music Generation with Explicit Bridges and Retrieval Augmentation},
author={Wang, Baisen and Zhuo, Le and Wang, Zhaokai and Bao, Chenxi and Wu, Chengjing and Nie, Xuecheng and Dai, Jiao and Han, Jizhong and Liao, Yue and Liu, Si},
journal={arXiv preprint arXiv:2412.09428},
year={2024}
}
@inproceedings{wang2025vision,
title={Vision-to-Music Generation: A Survey},
author={Wang, Zhaokai and Bao, Chenxi and Zhuo, Le and Han, Jingrui and Yue, Yang and Tang, Yihong and Huang, Victor Shea-Jay and Liao, Yue},
booktitle={ISMIR},
year={2025}
}
```
## Acknowledgements
Our code is based on [Compound Word Transformer](https://github.com/YatingMusic/compound-word-transformer).
`src/video2npz/visbeat3` is built upon [haofanwang/visbeat3](https://github.com/haofanwang/visbeat3) and the original [visbeat](http://abedavis.com/visualbeat/).
================================================
FILE: dataset/.gitkeep
================================================
================================================
FILE: exp/.gitkeep
================================================
================================================
FILE: inference/.gitkeep
================================================
================================================
FILE: logs/.gitkeep
================================================
================================================
FILE: py3_requirements.txt
================================================
tqdm
urllib3==1.26.9
six==1.16.0
threadpoolctl==2.1.0
h5py==2.10.0
matplotlib==3.3.4
numpy==1.23.1
miditoolkit==0.1.15
muspy==0.4.0
scikit_learn==1.0
torch==1.9.0
pytorch-fast-transformers
opencv-python==4.5.3.56
scikit-video==1.1.11
pypianoroll
scipy
bs4
librosa==0.6.2
imageio==2.9.0
requests
moviepy==1.0.3
termcolor==1.1.0
youtube-dl==2021.12.17
numba==0.48.0
pretty_midi
pyfluidsynth==1.2.5
note_seq
midi2audio
================================================
FILE: src/dictionary_mix.py
================================================
preset_event2word = {
"tempo" : {
0 : 0,
"CONTI" : 1,
"Tempo_0" : 2,
"Tempo_1" : 3,
"Tempo_2" : 4,
"Tempo_3" : 5,
"Tempo_4" : 6,
"Tempo_5" : 7,
"Tempo_6" : 8,
"Tempo_7" : 9,
"Tempo_8" : 10,
"Tempo_9" : 11,
"Tempo_10" : 12,
"Tempo_11" : 13,
"Tempo_12" : 14,
"Tempo_13" : 15,
"Tempo_14" : 16,
"Tempo_15" : 17,
"Tempo_16" : 18,
"Tempo_17" : 19,
"Tempo_18" : 20,
"Tempo_19" : 21,
"Tempo_20" : 22,
"Tempo_21" : 23,
"Tempo_22" : 24,
"Tempo_23" : 25,
"Tempo_24" : 26,
"Tempo_25" : 27,
"Tempo_26" : 28,
"Tempo_27" : 29,
"Tempo_28" : 30,
"Tempo_29" : 31,
"Tempo_30" : 32,
"Tempo_31" : 33,
"Tempo_32" : 34,
"Tempo_33" : 35,
"Tempo_34" : 36,
"Tempo_35" : 37,
"Tempo_36" : 38,
"Tempo_37" : 39,
"Tempo_38" : 40,
"Tempo_39" : 41,
"Tempo_40" : 42,
"Tempo_41" : 43,
"Tempo_42" : 44,
"Tempo_43" : 45,
"Tempo_44" : 46,
"Tempo_45" : 47,
"Tempo_46" : 48,
"Tempo_47" : 49,
"Tempo_48" : 50,
"Tempo_49" : 51,
"Tempo_50" : 52,
"Tempo_51" : 53,
"Tempo_52" : 54,
"Tempo_53" : 55,
"Tempo_54" : 56,
"Tempo_55" : 57,
"Tempo_56" : 58,
"Tempo_57" : 59,
"Tempo_58" : 60,
"Tempo_59" : 61,
"Tempo_60" : 62,
"Tempo_61" : 63,
"Tempo_62" : 64,
"Tempo_63" : 65,
"Tempo_64" : 66,
"Tempo_65" : 67,
"Tempo_66" : 68,
"Tempo_67" : 69,
"Tempo_68" : 70,
"Tempo_69" : 71,
"Tempo_70" : 72,
"Tempo_71" : 73,
"Tempo_72" : 74,
"Tempo_73" : 75,
"Tempo_74" : 76,
"Tempo_75" : 77,
"Tempo_76" : 78,
"Tempo_77" : 79,
"Tempo_78" : 80,
"Tempo_79" : 81,
"Tempo_80" : 82,
"Tempo_81" : 83,
"Tempo_82" : 84,
"Tempo_83" : 85,
"Tempo_84" : 86,
"Tempo_85" : 87,
"Tempo_86" : 88,
"Tempo_87" : 89,
"Tempo_88" : 90,
"Tempo_89" : 91,
"Tempo_90" : 92,
"Tempo_91" : 93,
"Tempo_92" : 94,
"Tempo_93" : 95,
"Tempo_94" : 96,
"Tempo_95" : 97,
"Tempo_96" : 98,
"Tempo_97" : 99,
"Tempo_98" : 100,
"Tempo_99" : 101,
"Tempo_100": 102,
"Tempo_101": 103,
"Tempo_102": 104,
"Tempo_103": 105,
"Tempo_104": 106,
"Tempo_105": 107,
"Tempo_106": 108,
"Tempo_107": 109,
"Tempo_108": 110,
"Tempo_109": 111,
"Tempo_110": 112,
"Tempo_111": 113,
"Tempo_112": 114,
"Tempo_113": 115,
"Tempo_114": 116,
"Tempo_115": 117,
"Tempo_116": 118,
"Tempo_117": 119,
"Tempo_118": 120,
"Tempo_119": 121,
"Tempo_120": 122,
"Tempo_121": 123,
"Tempo_122": 124,
"Tempo_123": 125,
"Tempo_124": 126,
"Tempo_125": 127,
"Tempo_126": 128,
"Tempo_127": 129,
"Tempo_128": 130,
"Tempo_129": 131,
"Tempo_130": 132,
"Tempo_131": 133,
"Tempo_132": 134,
"Tempo_133": 135,
"Tempo_134": 136,
"Tempo_135": 137,
"Tempo_136": 138,
"Tempo_137": 139,
"Tempo_138": 140,
"Tempo_139": 141,
"Tempo_140": 142,
"Tempo_141": 143,
"Tempo_142": 144,
"Tempo_143": 145,
"Tempo_144": 146,
"Tempo_145": 147,
"Tempo_146": 148,
"Tempo_147": 149,
"Tempo_148": 150,
"Tempo_149": 151,
"Tempo_150": 152,
"Tempo_151": 153,
"Tempo_152": 154,
"Tempo_153": 155,
"Tempo_154": 156,
"Tempo_155": 157,
"Tempo_156": 158,
"Tempo_157": 159,
"Tempo_158": 160,
"Tempo_159": 161,
"Tempo_160": 162,
"Tempo_161": 163,
"Tempo_162": 164,
"Tempo_163": 165,
"Tempo_164": 166,
"Tempo_165": 167,
"Tempo_166": 168,
"Tempo_167": 169,
"Tempo_168": 170,
"Tempo_169": 171,
"Tempo_170": 172,
"Tempo_171": 173,
"Tempo_172": 174,
"Tempo_173": 175,
"Tempo_174": 176,
"Tempo_175": 177,
"Tempo_176": 178,
"Tempo_177": 179,
"Tempo_178": 180,
"Tempo_179": 181,
"Tempo_180": 182,
"Tempo_181": 183,
"Tempo_182": 184,
"Tempo_183": 185,
"Tempo_184": 186,
"Tempo_185": 187,
"Tempo_186": 188,
"Tempo_187": 189,
"Tempo_188": 190,
"Tempo_189": 191,
"Tempo_190": 192,
"Tempo_191": 193,
"Tempo_192": 194,
"Tempo_193": 195,
"Tempo_194": 196,
"Tempo_195": 197,
"Tempo_196": 198,
"Tempo_197": 199,
"Tempo_198": 200,
"Tempo_199": 201,
"Tempo_200": 202,
"Tempo_201": 203,
"Tempo_202": 204,
"Tempo_203": 205,
"Tempo_204": 206,
"Tempo_205": 207,
"Tempo_206": 208,
"Tempo_207": 209,
"Tempo_208": 210,
"Tempo_209": 211,
"Tempo_210": 212,
"Tempo_211": 213,
"Tempo_212": 214,
"Tempo_213": 215,
"Tempo_214": 216,
"Tempo_215": 217,
"Tempo_216": 218,
"Tempo_217": 219,
"Tempo_218": 220,
"Tempo_219": 221,
"Tempo_220": 222,
"Tempo_221": 223,
"Tempo_222": 224,
"Tempo_223": 225,
"Tempo_224": 226,
"Tempo_225": 227,
"Tempo_226": 228,
"Tempo_227": 229,
"Tempo_228": 230,
"Tempo_229": 231,
"Tempo_230": 232,
"Tempo_231": 233,
"Tempo_232": 234,
"Tempo_233": 235,
"Tempo_234": 236,
"Tempo_235": 237,
"Tempo_236": 238,
"Tempo_237": 239,
"Tempo_238": 240,
"Tempo_239": 241,
"Tempo_240": 242,
"Tempo_241": 243,
"Tempo_242": 244,
"Tempo_243": 245,
"Tempo_244": 246,
"Tempo_245": 247,
"Tempo_246": 248,
"Tempo_247": 249,
"Tempo_248": 250,
"Tempo_249": 251,
"Tempo_250": 252,
"Tempo_251": 253,
"Tempo_252": 254,
"Tempo_253": 255,
"Tempo_254": 256,
"Tempo_255": 257,
"Tempo_256": 258,
"Tempo_257": 259,
"Tempo_258": 260,
"Tempo_259": 261,
"Tempo_260": 262,
"Tempo_261": 263,
"Tempo_262": 264,
"Tempo_263": 265,
"Tempo_264": 266,
"Tempo_265": 267,
"Tempo_266": 268,
"Tempo_267": 269,
"Tempo_268": 270,
"Tempo_269": 271,
"Tempo_270": 272,
"Tempo_271": 273,
"Tempo_272": 274,
"Tempo_273": 275,
"Tempo_274": 276,
"Tempo_275": 277,
"Tempo_276": 278,
"Tempo_277": 279,
"Tempo_278": 280,
"Tempo_279": 281,
"Tempo_280": 282,
"Tempo_281": 283,
"Tempo_282": 284,
"Tempo_283": 285,
"Tempo_284": 286,
"Tempo_285": 287,
"Tempo_286": 288,
"Tempo_287": 289,
"Tempo_288": 290,
"Tempo_289": 291,
"Tempo_290": 292,
"Tempo_291": 293,
"Tempo_292": 294,
"Tempo_293": 295,
"Tempo_294": 296,
"Tempo_295": 297,
"Tempo_296": 298,
"Tempo_297": 299,
"Tempo_298": 300,
"Tempo_299": 301,
"Tempo_300": 302,
"Tempo_301": 303,
"Tempo_302": 304,
"Tempo_303": 305,
"Tempo_304": 306,
"Tempo_305": 307,
"Tempo_306": 308,
"Tempo_307": 309,
"Tempo_308": 310,
"Tempo_309": 311,
"Tempo_310": 312,
"Tempo_311": 313,
"Tempo_312": 314,
"Tempo_313": 315,
"Tempo_314": 316,
"Tempo_315": 317,
"Tempo_316": 318,
"Tempo_317": 319,
"Tempo_318": 320,
"Tempo_319": 321,
"Tempo_320": 322,
"Tempo_321": 323,
"Tempo_322": 324,
"Tempo_323": 325,
"Tempo_324": 326,
"Tempo_325": 327,
"Tempo_326": 328,
"Tempo_327": 329,
"Tempo_328": 330,
"Tempo_329": 331,
"Tempo_330": 332,
"Tempo_331": 333,
"Tempo_332": 334,
"Tempo_333": 335,
"Tempo_334": 336,
"Tempo_335": 337,
"Tempo_336": 338,
"Tempo_337": 339,
"Tempo_338": 340,
"Tempo_339": 341,
"Tempo_340": 342,
"Tempo_341": 343,
"Tempo_342": 344,
"Tempo_343": 345,
"Tempo_344": 346,
"Tempo_345": 347,
"Tempo_346": 348,
"Tempo_347": 349,
"Tempo_348": 350,
"Tempo_349": 351,
"Tempo_350": 352,
"Tempo_351": 353,
"Tempo_352": 354,
"Tempo_353": 355,
"Tempo_354": 356,
"Tempo_355": 357,
"Tempo_356": 358,
"Tempo_357": 359,
"Tempo_358": 360,
"Tempo_359": 361,
"Tempo_360": 362
},
"chord" : {
0 : 0,
"CONTI": 1
},
"beat" : {
0 : 0,
"Beat_0" : 1,
"Beat_1" : 2,
"Beat_2" : 3,
"Beat_3" : 4,
"Beat_4" : 5,
"Beat_5" : 6,
"Beat_6" : 7,
"Beat_7" : 8,
"Beat_8" : 9,
"Beat_9" : 10,
"Beat_10": 11,
"Beat_11": 12,
"Beat_12": 13,
"Beat_13": 14,
"Beat_14": 15,
"Beat_15": 16,
"Bar" : 17,
},
"type" : {
"EOS" : 0,
"M" : 1,
"Note" : 2,
'Global': 3,
},
"instr_type": {
'None' : 0,
'Drums' : 1,
'Piano' : 2,
'Guitar' : 3,
'Bass' : 4,
'Strings': 5,
},
"key/genre" : {
"None" : 0,
'C' : 5,
'C#' : 6,
'D' : 7,
'D#' : 8,
'E' : 9,
'F' : 10,
'F#' : 11,
'G' : 12,
'G#' : 13,
'A' : 14,
'A#' : 15,
'B' : 16,
'c' : 17,
'c#' : 18,
'd' : 19,
'd#' : 20,
'e' : 21,
'f' : 22,
'f#' : 23,
'g' : 24,
'g#' : 25,
'a' : 26,
'a#' : 27,
'b' : 28,
'Pop' : 29,
'Rock' : 30,
'Country' : 31,
'Electronic': 32,
'Metal' : 33,
},
"pitch" : {
0 : 0,
"Note_Pitch_0" : 1,
"Note_Pitch_1" : 2,
"Note_Pitch_2" : 3,
"Note_Pitch_3" : 4,
"Note_Pitch_4" : 5,
"Note_Pitch_5" : 6,
"Note_Pitch_6" : 7,
"Note_Pitch_7" : 8,
"Note_Pitch_8" : 9,
"Note_Pitch_9" : 10,
"Note_Pitch_10" : 11,
"Note_Pitch_11" : 12,
"Note_Pitch_12" : 13,
"Note_Pitch_13" : 14,
"Note_Pitch_14" : 15,
"Note_Pitch_15" : 16,
"Note_Pitch_16" : 17,
"Note_Pitch_17" : 18,
"Note_Pitch_18" : 19,
"Note_Pitch_19" : 20,
"Note_Pitch_20" : 21,
"Note_Pitch_21" : 22,
"Note_Pitch_22" : 23,
"Note_Pitch_23" : 24,
"Note_Pitch_24" : 25,
"Note_Pitch_25" : 26,
"Note_Pitch_26" : 27,
"Note_Pitch_27" : 28,
"Note_Pitch_28" : 29,
"Note_Pitch_29" : 30,
"Note_Pitch_30" : 31,
"Note_Pitch_31" : 32,
"Note_Pitch_32" : 33,
"Note_Pitch_33" : 34,
"Note_Pitch_34" : 35,
"Note_Pitch_35" : 36,
"Note_Pitch_36" : 37,
"Note_Pitch_37" : 38,
"Note_Pitch_38" : 39,
"Note_Pitch_39" : 40,
"Note_Pitch_40" : 41,
"Note_Pitch_41" : 42,
"Note_Pitch_42" : 43,
"Note_Pitch_43" : 44,
"Note_Pitch_44" : 45,
"Note_Pitch_45" : 46,
"Note_Pitch_46" : 47,
"Note_Pitch_47" : 48,
"Note_Pitch_48" : 49,
"Note_Pitch_49" : 50,
"Note_Pitch_50" : 51,
"Note_Pitch_51" : 52,
"Note_Pitch_52" : 53,
"Note_Pitch_53" : 54,
"Note_Pitch_54" : 55,
"Note_Pitch_55" : 56,
"Note_Pitch_56" : 57,
"Note_Pitch_57" : 58,
"Note_Pitch_58" : 59,
"Note_Pitch_59" : 60,
"Note_Pitch_60" : 61,
"Note_Pitch_61" : 62,
"Note_Pitch_62" : 63,
"Note_Pitch_63" : 64,
"Note_Pitch_64" : 65,
"Note_Pitch_65" : 66,
"Note_Pitch_66" : 67,
"Note_Pitch_67" : 68,
"Note_Pitch_68" : 69,
"Note_Pitch_69" : 70,
"Note_Pitch_70" : 71,
"Note_Pitch_71" : 72,
"Note_Pitch_72" : 73,
"Note_Pitch_73" : 74,
"Note_Pitch_74" : 75,
"Note_Pitch_75" : 76,
"Note_Pitch_76" : 77,
"Note_Pitch_77" : 78,
"Note_Pitch_78" : 79,
"Note_Pitch_79" : 80,
"Note_Pitch_80" : 81,
"Note_Pitch_81" : 82,
"Note_Pitch_82" : 83,
"Note_Pitch_83" : 84,
"Note_Pitch_84" : 85,
"Note_Pitch_85" : 86,
"Note_Pitch_86" : 87,
"Note_Pitch_87" : 88,
"Note_Pitch_88" : 89,
"Note_Pitch_89" : 90,
"Note_Pitch_90" : 91,
"Note_Pitch_91" : 92,
"Note_Pitch_92" : 93,
"Note_Pitch_93" : 94,
"Note_Pitch_94" : 95,
"Note_Pitch_95" : 96,
"Note_Pitch_96" : 97,
"Note_Pitch_97" : 98,
"Note_Pitch_98" : 99,
"Note_Pitch_99" : 100,
"Note_Pitch_100": 101,
"Note_Pitch_101": 102,
"Note_Pitch_102": 103,
"Note_Pitch_103": 104,
"Note_Pitch_104": 105,
"Note_Pitch_105": 106,
"Note_Pitch_106": 107,
"Note_Pitch_107": 108,
"Note_Pitch_108": 109,
"Note_Pitch_109": 110,
"Note_Pitch_110": 111,
"Note_Pitch_111": 112,
"Note_Pitch_112": 113,
"Note_Pitch_113": 114,
"Note_Pitch_114": 115,
"Note_Pitch_115": 116,
"Note_Pitch_116": 117,
"Note_Pitch_117": 118,
"Note_Pitch_118": 119,
"Note_Pitch_119": 120,
"Note_Pitch_120": 121,
"Note_Pitch_121": 122,
"Note_Pitch_122": 123,
"Note_Pitch_123": 124,
"Note_Pitch_124": 125,
"Note_Pitch_125": 126,
"Note_Pitch_126": 127,
"Note_Pitch_127": 128,
},
"duration" : {
0 : 0,
"Note_Duration_0" : 1,
"Note_Duration_120" : 2,
"Note_Duration_240" : 3,
"Note_Duration_360" : 4,
"Note_Duration_480" : 5,
"Note_Duration_600" : 6,
"Note_Duration_720" : 7,
"Note_Duration_840" : 8,
"Note_Duration_960" : 9,
"Note_Duration_1080": 10,
"Note_Duration_1200": 11,
"Note_Duration_1320": 12,
"Note_Duration_1440": 13,
"Note_Duration_1560": 14,
"Note_Duration_1680": 15,
"Note_Duration_1800": 16,
"Note_Duration_1920": 17
},
"velocity" : {
0 : 0,
"Note_Velocity_0" : 1,
"Note_Velocity_1" : 2,
"Note_Velocity_2" : 3,
"Note_Velocity_3" : 4,
"Note_Velocity_4" : 5,
"Note_Velocity_5" : 6,
"Note_Velocity_6" : 7,
"Note_Velocity_7" : 8,
"Note_Velocity_8" : 9,
"Note_Velocity_9" : 10,
"Note_Velocity_10" : 11,
"Note_Velocity_11" : 12,
"Note_Velocity_12" : 13,
"Note_Velocity_13" : 14,
"Note_Velocity_14" : 15,
"Note_Velocity_15" : 16,
"Note_Velocity_16" : 17,
"Note_Velocity_17" : 18,
"Note_Velocity_18" : 19,
"Note_Velocity_19" : 20,
"Note_Velocity_20" : 21,
"Note_Velocity_21" : 22,
"Note_Velocity_22" : 23,
"Note_Velocity_23" : 24,
"Note_Velocity_24" : 25,
"Note_Velocity_25" : 26,
"Note_Velocity_26" : 27,
"Note_Velocity_27" : 28,
"Note_Velocity_28" : 29,
"Note_Velocity_29" : 30,
"Note_Velocity_30" : 31,
"Note_Velocity_31" : 32,
"Note_Velocity_32" : 33,
"Note_Velocity_33" : 34,
"Note_Velocity_34" : 35,
"Note_Velocity_35" : 36,
"Note_Velocity_36" : 37,
"Note_Velocity_37" : 38,
"Note_Velocity_38" : 39,
"Note_Velocity_39" : 40,
"Note_Velocity_40" : 41,
"Note_Velocity_41" : 42,
"Note_Velocity_42" : 43,
"Note_Velocity_43" : 44,
"Note_Velocity_44" : 45,
"Note_Velocity_45" : 46,
"Note_Velocity_46" : 47,
"Note_Velocity_47" : 48,
"Note_Velocity_48" : 49,
"Note_Velocity_49" : 50,
"Note_Velocity_50" : 51,
"Note_Velocity_51" : 52,
"Note_Velocity_52" : 53,
"Note_Velocity_53" : 54,
"Note_Velocity_54" : 55,
"Note_Velocity_55" : 56,
"Note_Velocity_56" : 57,
"Note_Velocity_57" : 58,
"Note_Velocity_58" : 59,
"Note_Velocity_59" : 60,
"Note_Velocity_60" : 61,
"Note_Velocity_61" : 62,
"Note_Velocity_62" : 63,
"Note_Velocity_63" : 64,
"Note_Velocity_64" : 65,
"Note_Velocity_65" : 66,
"Note_Velocity_66" : 67,
"Note_Velocity_67" : 68,
"Note_Velocity_68" : 69,
"Note_Velocity_69" : 70,
"Note_Velocity_70" : 71,
"Note_Velocity_71" : 72,
"Note_Velocity_72" : 73,
"Note_Velocity_73" : 74,
"Note_Velocity_74" : 75,
"Note_Velocity_75" : 76,
"Note_Velocity_76" : 77,
"Note_Velocity_77" : 78,
"Note_Velocity_78" : 79,
"Note_Velocity_79" : 80,
"Note_Velocity_80" : 81,
"Note_Velocity_81" : 82,
"Note_Velocity_82" : 83,
"Note_Velocity_83" : 84,
"Note_Velocity_84" : 85,
"Note_Velocity_85" : 86,
"Note_Velocity_86" : 87,
"Note_Velocity_87" : 88,
"Note_Velocity_88" : 89,
"Note_Velocity_89" : 90,
"Note_Velocity_90" : 91,
"Note_Velocity_91" : 92,
"Note_Velocity_92" : 93,
"Note_Velocity_93" : 94,
"Note_Velocity_94" : 95,
"Note_Velocity_95" : 96,
"Note_Velocity_96" : 97,
"Note_Velocity_97" : 98,
"Note_Velocity_98" : 99,
"Note_Velocity_99" : 100,
"Note_Velocity_100": 101,
"Note_Velocity_101": 102,
"Note_Velocity_102": 103,
"Note_Velocity_103": 104,
"Note_Velocity_104": 105,
"Note_Velocity_105": 106,
"Note_Velocity_106": 107,
"Note_Velocity_107": 108,
"Note_Velocity_108": 109,
"Note_Velocity_109": 110,
"Note_Velocity_110": 111,
"Note_Velocity_111": 112,
"Note_Velocity_112": 113,
"Note_Velocity_113": 114,
"Note_Velocity_114": 115,
"Note_Velocity_115": 116,
"Note_Velocity_116": 117,
"Note_Velocity_117": 118,
"Note_Velocity_118": 119,
"Note_Velocity_119": 120,
"Note_Velocity_120": 121,
"Note_Velocity_121": 122,
"Note_Velocity_122": 123,
"Note_Velocity_123": 124,
"Note_Velocity_124": 125,
"Note_Velocity_125": 126,
"Note_Velocity_126": 127,
"Note_Velocity_127": 128,
"Note_Velocity_128": 129,
"Note_Velocity_129": 130,
"Note_Velocity_130": 131,
"Note_Velocity_131": 132,
"Note_Velocity_132": 133,
"Note_Velocity_133": 134,
"Note_Velocity_134": 135,
"Note_Velocity_135": 136,
"Note_Velocity_136": 137,
"Note_Velocity_137": 138,
"Note_Velocity_138": 139,
"Note_Velocity_139": 140,
"Note_Velocity_140": 141,
"Note_Velocity_141": 142,
"Note_Velocity_142": 143,
"Note_Velocity_143": 144,
"Note_Velocity_144": 145,
"Note_Velocity_145": 146,
"Note_Velocity_146": 147,
"Note_Velocity_147": 148,
"Note_Velocity_148": 149,
"Note_Velocity_149": 150,
"Note_Velocity_150": 151,
"Note_Velocity_151": 152,
"Note_Velocity_152": 153,
"Note_Velocity_153": 154,
"Note_Velocity_154": 155,
"Note_Velocity_155": 156,
"Note_Velocity_156": 157,
"Note_Velocity_157": 158,
"Note_Velocity_158": 159,
"Note_Velocity_159": 160,
"Note_Velocity_160": 161,
"Note_Velocity_161": 162,
"Note_Velocity_162": 163,
"Note_Velocity_163": 164,
"Note_Velocity_164": 165,
"Note_Velocity_165": 166,
"Note_Velocity_166": 167,
"Note_Velocity_167": 168,
"Note_Velocity_168": 169,
"Note_Velocity_169": 170,
"Note_Velocity_170": 171,
"Note_Velocity_171": 172,
"Note_Velocity_172": 173,
"Note_Velocity_173": 174,
"Note_Velocity_174": 175,
"Note_Velocity_175": 176,
"Note_Velocity_176": 177,
"Note_Velocity_177": 178,
"Note_Velocity_178": 179,
"Note_Velocity_179": 180,
"Note_Velocity_180": 181,
"Note_Velocity_181": 182,
"Note_Velocity_182": 183,
"Note_Velocity_183": 184,
"Note_Velocity_184": 185,
"Note_Velocity_185": 186,
"Note_Velocity_186": 187,
"Note_Velocity_187": 188,
"Note_Velocity_188": 189,
"Note_Velocity_189": 190,
"Note_Velocity_190": 191,
"Note_Velocity_191": 192,
"Note_Velocity_192": 193,
"Note_Velocity_193": 194,
"Note_Velocity_194": 195,
"Note_Velocity_195": 196,
"Note_Velocity_196": 197,
"Note_Velocity_197": 198,
"Note_Velocity_198": 199,
"Note_Velocity_199": 200,
"Note_Velocity_200": 201,
"Note_Velocity_201": 202,
"Note_Velocity_202": 203,
"Note_Velocity_203": 204,
"Note_Velocity_204": 205,
"Note_Velocity_205": 206,
"Note_Velocity_206": 207,
"Note_Velocity_207": 208,
"Note_Velocity_208": 209,
"Note_Velocity_209": 210,
"Note_Velocity_210": 211,
"Note_Velocity_211": 212,
"Note_Velocity_212": 213,
"Note_Velocity_213": 214,
"Note_Velocity_214": 215,
"Note_Velocity_215": 216,
"Note_Velocity_216": 217,
"Note_Velocity_217": 218,
"Note_Velocity_218": 219,
"Note_Velocity_219": 220,
"Note_Velocity_220": 221,
"Note_Velocity_221": 222,
"Note_Velocity_222": 223,
"Note_Velocity_223": 224,
"Note_Velocity_224": 225,
"Note_Velocity_225": 226,
"Note_Velocity_226": 227,
"Note_Velocity_227": 228,
"Note_Velocity_228": 229,
"Note_Velocity_229": 230,
"Note_Velocity_230": 231,
"Note_Velocity_231": 232,
"Note_Velocity_232": 233,
"Note_Velocity_233": 234,
"Note_Velocity_234": 235,
"Note_Velocity_235": 236,
"Note_Velocity_236": 237,
"Note_Velocity_237": 238,
"Note_Velocity_238": 239,
"Note_Velocity_239": 240,
"Note_Velocity_240": 241,
"Note_Velocity_241": 242,
"Note_Velocity_242": 243,
"Note_Velocity_243": 244,
"Note_Velocity_244": 245,
"Note_Velocity_245": 246,
"Note_Velocity_246": 247,
"Note_Velocity_247": 248,
"Note_Velocity_248": 249,
"Note_Velocity_249": 250,
"Note_Velocity_250": 251,
"Note_Velocity_251": 252,
"Note_Velocity_252": 253,
"Note_Velocity_253": 254,
"Note_Velocity_254": 255,
"Note_Velocity_255": 256,
"Note_Velocity_256": 257,
"Note_Velocity_257": 258,
"Note_Velocity_258": 259,
"Note_Velocity_259": 260,
"Note_Velocity_260": 261,
"Note_Velocity_261": 262,
"Note_Velocity_262": 263,
"Note_Velocity_263": 264,
"Note_Velocity_264": 265,
"Note_Velocity_265": 266,
"Note_Velocity_266": 267,
"Note_Velocity_267": 268,
"Note_Velocity_268": 269,
"Note_Velocity_269": 270,
"Note_Velocity_270": 271,
"Note_Velocity_271": 272,
"Note_Velocity_272": 273,
"Note_Velocity_273": 274,
"Note_Velocity_274": 275,
"Note_Velocity_275": 276,
"Note_Velocity_276": 277,
"Note_Velocity_277": 278,
"Note_Velocity_278": 279,
"Note_Velocity_279": 280,
"Note_Velocity_280": 281,
"Note_Velocity_281": 282,
"Note_Velocity_282": 283,
"Note_Velocity_283": 284,
"Note_Velocity_284": 285,
"Note_Velocity_285": 286,
"Note_Velocity_286": 287,
"Note_Velocity_287": 288,
"Note_Velocity_288": 289,
"Note_Velocity_289": 290,
"Note_Velocity_290": 291,
"Note_Velocity_291": 292,
"Note_Velocity_292": 293,
"Note_Velocity_293": 294,
"Note_Velocity_294": 295,
"Note_Velocity_295": 296,
"Note_Velocity_296": 297,
"Note_Velocity_297": 298,
"Note_Velocity_298": 299,
"Note_Velocity_299": 300,
"Note_Velocity_300": 301,
"Note_Velocity_301": 302,
"Note_Velocity_302": 303,
"Note_Velocity_303": 304,
"Note_Velocity_304": 305,
"Note_Velocity_305": 306,
"Note_Velocity_306": 307,
"Note_Velocity_307": 308,
"Note_Velocity_308": 309,
"Note_Velocity_309": 310,
"Note_Velocity_310": 311,
"Note_Velocity_311": 312,
"Note_Velocity_312": 313,
"Note_Velocity_313": 314,
"Note_Velocity_314": 315,
"Note_Velocity_315": 316,
"Note_Velocity_316": 317,
"Note_Velocity_317": 318,
"Note_Velocity_318": 319,
"Note_Velocity_319": 320,
"Note_Velocity_320": 321,
"Note_Velocity_321": 322,
"Note_Velocity_322": 323,
"Note_Velocity_323": 324,
"Note_Velocity_324": 325,
"Note_Velocity_325": 326,
"Note_Velocity_326": 327,
"Note_Velocity_327": 328,
"Note_Velocity_328": 329,
"Note_Velocity_329": 330,
"Note_Velocity_330": 331,
"Note_Velocity_331": 332,
"Note_Velocity_332": 333,
"Note_Velocity_333": 334,
"Note_Velocity_334": 335,
"Note_Velocity_335": 336,
"Note_Velocity_336": 337,
"Note_Velocity_337": 338,
"Note_Velocity_338": 339,
"Note_Velocity_339": 340,
"Note_Velocity_340": 341,
"Note_Velocity_341": 342,
"Note_Velocity_342": 343,
"Note_Velocity_343": 344,
"Note_Velocity_344": 345,
"Note_Velocity_345": 346,
"Note_Velocity_346": 347,
"Note_Velocity_347": 348,
"Note_Velocity_348": 349,
"Note_Velocity_349": 350,
"Note_Velocity_350": 351,
"Note_Velocity_351": 352,
"Note_Velocity_352": 353,
"Note_Velocity_353": 354,
"Note_Velocity_354": 355,
"Note_Velocity_355": 356,
"Note_Velocity_356": 357,
"Note_Velocity_357": 358,
"Note_Velocity_358": 359,
"Note_Velocity_359": 360,
"Note_Velocity_360": 361
},
"boundary" : {
0 : 0,
"None" : 1,
"Boundary": 2,
},
# 'density': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
'variance' : [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
}
preset_word2event = {
"tempo" : {
0 : 0,
1 : "CONTI",
2 : "Tempo_0",
3 : "Tempo_1",
4 : "Tempo_2",
5 : "Tempo_3",
6 : "Tempo_4",
7 : "Tempo_5",
8 : "Tempo_6",
9 : "Tempo_7",
10 : "Tempo_8",
11 : "Tempo_9",
12 : "Tempo_10",
13 : "Tempo_11",
14 : "Tempo_12",
15 : "Tempo_13",
16 : "Tempo_14",
17 : "Tempo_15",
18 : "Tempo_16",
19 : "Tempo_17",
20 : "Tempo_18",
21 : "Tempo_19",
22 : "Tempo_20",
23 : "Tempo_21",
24 : "Tempo_22",
25 : "Tempo_23",
26 : "Tempo_24",
27 : "Tempo_25",
28 : "Tempo_26",
29 : "Tempo_27",
30 : "Tempo_28",
31 : "Tempo_29",
32 : "Tempo_30",
33 : "Tempo_31",
34 : "Tempo_32",
35 : "Tempo_33",
36 : "Tempo_34",
37 : "Tempo_35",
38 : "Tempo_36",
39 : "Tempo_37",
40 : "Tempo_38",
41 : "Tempo_39",
42 : "Tempo_40",
43 : "Tempo_41",
44 : "Tempo_42",
45 : "Tempo_43",
46 : "Tempo_44",
47 : "Tempo_45",
48 : "Tempo_46",
49 : "Tempo_47",
50 : "Tempo_48",
51 : "Tempo_49",
52 : "Tempo_50",
53 : "Tempo_51",
54 : "Tempo_52",
55 : "Tempo_53",
56 : "Tempo_54",
57 : "Tempo_55",
58 : "Tempo_56",
59 : "Tempo_57",
60 : "Tempo_58",
61 : "Tempo_59",
62 : "Tempo_60",
63 : "Tempo_61",
64 : "Tempo_62",
65 : "Tempo_63",
66 : "Tempo_64",
67 : "Tempo_65",
68 : "Tempo_66",
69 : "Tempo_67",
70 : "Tempo_68",
71 : "Tempo_69",
72 : "Tempo_70",
73 : "Tempo_71",
74 : "Tempo_72",
75 : "Tempo_73",
76 : "Tempo_74",
77 : "Tempo_75",
78 : "Tempo_76",
79 : "Tempo_77",
80 : "Tempo_78",
81 : "Tempo_79",
82 : "Tempo_80",
83 : "Tempo_81",
84 : "Tempo_82",
85 : "Tempo_83",
86 : "Tempo_84",
87 : "Tempo_85",
88 : "Tempo_86",
89 : "Tempo_87",
90 : "Tempo_88",
91 : "Tempo_89",
92 : "Tempo_90",
93 : "Tempo_91",
94 : "Tempo_92",
95 : "Tempo_93",
96 : "Tempo_94",
97 : "Tempo_95",
98 : "Tempo_96",
99 : "Tempo_97",
100: "Tempo_98",
101: "Tempo_99",
102: "Tempo_100",
103: "Tempo_101",
104: "Tempo_102",
105: "Tempo_103",
106: "Tempo_104",
107: "Tempo_105",
108: "Tempo_106",
109: "Tempo_107",
110: "Tempo_108",
111: "Tempo_109",
112: "Tempo_110",
113: "Tempo_111",
114: "Tempo_112",
115: "Tempo_113",
116: "Tempo_114",
117: "Tempo_115",
118: "Tempo_116",
119: "Tempo_117",
120: "Tempo_118",
121: "Tempo_119",
122: "Tempo_120",
123: "Tempo_121",
124: "Tempo_122",
125: "Tempo_123",
126: "Tempo_124",
127: "Tempo_125",
128: "Tempo_126",
129: "Tempo_127",
130: "Tempo_128",
131: "Tempo_129",
132: "Tempo_130",
133: "Tempo_131",
134: "Tempo_132",
135: "Tempo_133",
136: "Tempo_134",
137: "Tempo_135",
138: "Tempo_136",
139: "Tempo_137",
140: "Tempo_138",
141: "Tempo_139",
142: "Tempo_140",
143: "Tempo_141",
144: "Tempo_142",
145: "Tempo_143",
146: "Tempo_144",
147: "Tempo_145",
148: "Tempo_146",
149: "Tempo_147",
150: "Tempo_148",
151: "Tempo_149",
152: "Tempo_150",
153: "Tempo_151",
154: "Tempo_152",
155: "Tempo_153",
156: "Tempo_154",
157: "Tempo_155",
158: "Tempo_156",
159: "Tempo_157",
160: "Tempo_158",
161: "Tempo_159",
162: "Tempo_160",
163: "Tempo_161",
164: "Tempo_162",
165: "Tempo_163",
166: "Tempo_164",
167: "Tempo_165",
168: "Tempo_166",
169: "Tempo_167",
170: "Tempo_168",
171: "Tempo_169",
172: "Tempo_170",
173: "Tempo_171",
174: "Tempo_172",
175: "Tempo_173",
176: "Tempo_174",
177: "Tempo_175",
178: "Tempo_176",
179: "Tempo_177",
180: "Tempo_178",
181: "Tempo_179",
182: "Tempo_180",
183: "Tempo_181",
184: "Tempo_182",
185: "Tempo_183",
186: "Tempo_184",
187: "Tempo_185",
188: "Tempo_186",
189: "Tempo_187",
190: "Tempo_188",
191: "Tempo_189",
192: "Tempo_190",
193: "Tempo_191",
194: "Tempo_192",
195: "Tempo_193",
196: "Tempo_194",
197: "Tempo_195",
198: "Tempo_196",
199: "Tempo_197",
200: "Tempo_198",
201: "Tempo_199",
202: "Tempo_200",
203: "Tempo_201",
204: "Tempo_202",
205: "Tempo_203",
206: "Tempo_204",
207: "Tempo_205",
208: "Tempo_206",
209: "Tempo_207",
210: "Tempo_208",
211: "Tempo_209",
212: "Tempo_210",
213: "Tempo_211",
214: "Tempo_212",
215: "Tempo_213",
216: "Tempo_214",
217: "Tempo_215",
218: "Tempo_216",
219: "Tempo_217",
220: "Tempo_218",
221: "Tempo_219",
222: "Tempo_220",
223: "Tempo_221",
224: "Tempo_222",
225: "Tempo_223",
226: "Tempo_224",
227: "Tempo_225",
228: "Tempo_226",
229: "Tempo_227",
230: "Tempo_228",
231: "Tempo_229",
232: "Tempo_230",
233: "Tempo_231",
234: "Tempo_232",
235: "Tempo_233",
236: "Tempo_234",
237: "Tempo_235",
238: "Tempo_236",
239: "Tempo_237",
240: "Tempo_238",
241: "Tempo_239",
242: "Tempo_240",
243: "Tempo_241",
244: "Tempo_242",
245: "Tempo_243",
246: "Tempo_244",
247: "Tempo_245",
248: "Tempo_246",
249: "Tempo_247",
250: "Tempo_248",
251: "Tempo_249",
252: "Tempo_250",
253: "Tempo_251",
254: "Tempo_252",
255: "Tempo_253",
256: "Tempo_254",
257: "Tempo_255",
258: "Tempo_256",
259: "Tempo_257",
260: "Tempo_258",
261: "Tempo_259",
262: "Tempo_260",
263: "Tempo_261",
264: "Tempo_262",
265: "Tempo_263",
266: "Tempo_264",
267: "Tempo_265",
268: "Tempo_266",
269: "Tempo_267",
270: "Tempo_268",
271: "Tempo_269",
272: "Tempo_270",
273: "Tempo_271",
274: "Tempo_272",
275: "Tempo_273",
276: "Tempo_274",
277: "Tempo_275",
278: "Tempo_276",
279: "Tempo_277",
280: "Tempo_278",
281: "Tempo_279",
282: "Tempo_280",
283: "Tempo_281",
284: "Tempo_282",
285: "Tempo_283",
286: "Tempo_284",
287: "Tempo_285",
288: "Tempo_286",
289: "Tempo_287",
290: "Tempo_288",
291: "Tempo_289",
292: "Tempo_290",
293: "Tempo_291",
294: "Tempo_292",
295: "Tempo_293",
296: "Tempo_294",
297: "Tempo_295",
298: "Tempo_296",
299: "Tempo_297",
300: "Tempo_298",
301: "Tempo_299",
302: "Tempo_300",
303: "Tempo_301",
304: "Tempo_302",
305: "Tempo_303",
306: "Tempo_304",
307: "Tempo_305",
308: "Tempo_306",
309: "Tempo_307",
310: "Tempo_308",
311: "Tempo_309",
312: "Tempo_310",
313: "Tempo_311",
314: "Tempo_312",
315: "Tempo_313",
316: "Tempo_314",
317: "Tempo_315",
318: "Tempo_316",
319: "Tempo_317",
320: "Tempo_318",
321: "Tempo_319",
322: "Tempo_320",
323: "Tempo_321",
324: "Tempo_322",
325: "Tempo_323",
326: "Tempo_324",
327: "Tempo_325",
328: "Tempo_326",
329: "Tempo_327",
330: "Tempo_328",
331: "Tempo_329",
332: "Tempo_330",
333: "Tempo_331",
334: "Tempo_332",
335: "Tempo_333",
336: "Tempo_334",
337: "Tempo_335",
338: "Tempo_336",
339: "Tempo_337",
340: "Tempo_338",
341: "Tempo_339",
342: "Tempo_340",
343: "Tempo_341",
344: "Tempo_342",
345: "Tempo_343",
346: "Tempo_344",
347: "Tempo_345",
348: "Tempo_346",
349: "Tempo_347",
350: "Tempo_348",
351: "Tempo_349",
352: "Tempo_350",
353: "Tempo_351",
354: "Tempo_352",
355: "Tempo_353",
356: "Tempo_354",
357: "Tempo_355",
358: "Tempo_356",
359: "Tempo_357",
360: "Tempo_358",
361: "Tempo_359",
362: "Tempo_360"
},
"chord" : {
0: "0",
1: "CONTI"
},
"bar-beat" : {
0 : 0,
1 : "Bar",
2 : "Beat_0",
3 : "Beat_1",
4 : "Beat_2",
5 : "Beat_3",
6 : "Beat_4",
7 : "Beat_5",
8 : "Beat_6",
9 : "Beat_7",
10: "Beat_8",
11: "Beat_9",
12: "Beat_10",
13: "Beat_11",
14: "Beat_12",
15: "Beat_13",
16: "Beat_14",
17: "Beat_15"
},
"type" : {
0: "EOS",
1: "Metrical",
2: "Note",
3: "Seg",
},
"instr_type": {
0: 'None',
1: 'Drums',
2: 'Piano',
3: 'Guitar',
4: 'Bass',
5: 'Strings',
},
"pitch" : {
0 : 0,
1 : "Note_Pitch_0",
2 : "Note_Pitch_1",
3 : "Note_Pitch_2",
4 : "Note_Pitch_3",
5 : "Note_Pitch_4",
6 : "Note_Pitch_5",
7 : "Note_Pitch_6",
8 : "Note_Pitch_7",
9 : "Note_Pitch_8",
10 : "Note_Pitch_9",
11 : "Note_Pitch_10",
12 : "Note_Pitch_11",
13 : "Note_Pitch_12",
14 : "Note_Pitch_13",
15 : "Note_Pitch_14",
16 : "Note_Pitch_15",
17 : "Note_Pitch_16",
18 : "Note_Pitch_17",
19 : "Note_Pitch_18",
20 : "Note_Pitch_19",
21 : "Note_Pitch_20",
22 : "Note_Pitch_21",
23 : "Note_Pitch_22",
24 : "Note_Pitch_23",
25 : "Note_Pitch_24",
26 : "Note_Pitch_25",
27 : "Note_Pitch_26",
28 : "Note_Pitch_27",
29 : "Note_Pitch_28",
30 : "Note_Pitch_29",
31 : "Note_Pitch_30",
32 : "Note_Pitch_31",
33 : "Note_Pitch_32",
34 : "Note_Pitch_33",
35 : "Note_Pitch_34",
36 : "Note_Pitch_35",
37 : "Note_Pitch_36",
38 : "Note_Pitch_37",
39 : "Note_Pitch_38",
40 : "Note_Pitch_39",
41 : "Note_Pitch_40",
42 : "Note_Pitch_41",
43 : "Note_Pitch_42",
44 : "Note_Pitch_43",
45 : "Note_Pitch_44",
46 : "Note_Pitch_45",
47 : "Note_Pitch_46",
48 : "Note_Pitch_47",
49 : "Note_Pitch_48",
50 : "Note_Pitch_49",
51 : "Note_Pitch_50",
52 : "Note_Pitch_51",
53 : "Note_Pitch_52",
54 : "Note_Pitch_53",
55 : "Note_Pitch_54",
56 : "Note_Pitch_55",
57 : "Note_Pitch_56",
58 : "Note_Pitch_57",
59 : "Note_Pitch_58",
60 : "Note_Pitch_59",
61 : "Note_Pitch_60",
62 : "Note_Pitch_61",
63 : "Note_Pitch_62",
64 : "Note_Pitch_63",
65 : "Note_Pitch_64",
66 : "Note_Pitch_65",
67 : "Note_Pitch_66",
68 : "Note_Pitch_67",
69 : "Note_Pitch_68",
70 : "Note_Pitch_69",
71 : "Note_Pitch_70",
72 : "Note_Pitch_71",
73 : "Note_Pitch_72",
74 : "Note_Pitch_73",
75 : "Note_Pitch_74",
76 : "Note_Pitch_75",
77 : "Note_Pitch_76",
78 : "Note_Pitch_77",
79 : "Note_Pitch_78",
80 : "Note_Pitch_79",
81 : "Note_Pitch_80",
82 : "Note_Pitch_81",
83 : "Note_Pitch_82",
84 : "Note_Pitch_83",
85 : "Note_Pitch_84",
86 : "Note_Pitch_85",
87 : "Note_Pitch_86",
88 : "Note_Pitch_87",
89 : "Note_Pitch_88",
90 : "Note_Pitch_89",
91 : "Note_Pitch_90",
92 : "Note_Pitch_91",
93 : "Note_Pitch_92",
94 : "Note_Pitch_93",
95 : "Note_Pitch_94",
96 : "Note_Pitch_95",
97 : "Note_Pitch_96",
98 : "Note_Pitch_97",
99 : "Note_Pitch_98",
100: "Note_Pitch_99",
101: "Note_Pitch_100",
102: "Note_Pitch_101",
103: "Note_Pitch_102",
104: "Note_Pitch_103",
105: "Note_Pitch_104",
106: "Note_Pitch_105",
107: "Note_Pitch_106",
108: "Note_Pitch_107",
109: "Note_Pitch_108",
110: "Note_Pitch_109",
111: "Note_Pitch_110",
112: "Note_Pitch_111",
113: "Note_Pitch_112",
114: "Note_Pitch_113",
115: "Note_Pitch_114",
116: "Note_Pitch_115",
117: "Note_Pitch_116",
118: "Note_Pitch_117",
119: "Note_Pitch_118",
120: "Note_Pitch_119",
121: "Note_Pitch_120",
122: "Note_Pitch_121",
123: "Note_Pitch_122",
124: "Note_Pitch_123",
125: "Note_Pitch_124",
126: "Note_Pitch_125",
127: "Note_Pitch_126",
128: "Note_Pitch_127",
129: "Note_Pitch_128",
},
"duration" : {
0 : 0,
1 : "Note_Duration_0",
2 : "Note_Duration_120",
3 : "Note_Duration_240",
4 : "Note_Duration_360",
5 : "Note_Duration_480",
6 : "Note_Duration_600",
7 : "Note_Duration_720",
8 : "Note_Duration_840",
9 : "Note_Duration_960",
10: "Note_Duration_1080",
11: "Note_Duration_1200",
12: "Note_Duration_1320",
13: "Note_Duration_1440",
14: "Note_Duration_1560",
15: "Note_Duration_1680",
16: "Note_Duration_1800",
17: "Note_Duration_1920"
},
"velocity" : {
0 : 0,
1 : "Note_Velocity_0",
2 : "Note_Velocity_1",
3 : "Note_Velocity_2",
4 : "Note_Velocity_3",
5 : "Note_Velocity_4",
6 : "Note_Velocity_5",
7 : "Note_Velocity_6",
8 : "Note_Velocity_7",
9 : "Note_Velocity_8",
10 : "Note_Velocity_9",
11 : "Note_Velocity_10",
12 : "Note_Velocity_11",
13 : "Note_Velocity_12",
14 : "Note_Velocity_13",
15 : "Note_Velocity_14",
16 : "Note_Velocity_15",
17 : "Note_Velocity_16",
18 : "Note_Velocity_17",
19 : "Note_Velocity_18",
20 : "Note_Velocity_19",
21 : "Note_Velocity_20",
22 : "Note_Velocity_21",
23 : "Note_Velocity_22",
24 : "Note_Velocity_23",
25 : "Note_Velocity_24",
26 : "Note_Velocity_25",
27 : "Note_Velocity_26",
28 : "Note_Velocity_27",
29 : "Note_Velocity_28",
30 : "Note_Velocity_29",
31 : "Note_Velocity_30",
32 : "Note_Velocity_31",
33 : "Note_Velocity_32",
34 : "Note_Velocity_33",
35 : "Note_Velocity_34",
36 : "Note_Velocity_35",
37 : "Note_Velocity_36",
38 : "Note_Velocity_37",
39 : "Note_Velocity_38",
40 : "Note_Velocity_39",
41 : "Note_Velocity_40",
42 : "Note_Velocity_41",
43 : "Note_Velocity_42",
44 : "Note_Velocity_43",
45 : "Note_Velocity_44",
46 : "Note_Velocity_45",
47 : "Note_Velocity_46",
48 : "Note_Velocity_47",
49 : "Note_Velocity_48",
50 : "Note_Velocity_49",
51 : "Note_Velocity_50",
52 : "Note_Velocity_51",
53 : "Note_Velocity_52",
54 : "Note_Velocity_53",
55 : "Note_Velocity_54",
56 : "Note_Velocity_55",
57 : "Note_Velocity_56",
58 : "Note_Velocity_57",
59 : "Note_Velocity_58",
60 : "Note_Velocity_59",
61 : "Note_Velocity_60",
62 : "Note_Velocity_61",
63 : "Note_Velocity_62",
64 : "Note_Velocity_63",
65 : "Note_Velocity_64",
66 : "Note_Velocity_65",
67 : "Note_Velocity_66",
68 : "Note_Velocity_67",
69 : "Note_Velocity_68",
70 : "Note_Velocity_69",
71 : "Note_Velocity_70",
72 : "Note_Velocity_71",
73 : "Note_Velocity_72",
74 : "Note_Velocity_73",
75 : "Note_Velocity_74",
76 : "Note_Velocity_75",
77 : "Note_Velocity_76",
78 : "Note_Velocity_77",
79 : "Note_Velocity_78",
80 : "Note_Velocity_79",
81 : "Note_Velocity_80",
82 : "Note_Velocity_81",
83 : "Note_Velocity_82",
84 : "Note_Velocity_83",
85 : "Note_Velocity_84",
86 : "Note_Velocity_85",
87 : "Note_Velocity_86",
88 : "Note_Velocity_87",
89 : "Note_Velocity_88",
90 : "Note_Velocity_89",
91 : "Note_Velocity_90",
92 : "Note_Velocity_91",
93 : "Note_Velocity_92",
94 : "Note_Velocity_93",
95 : "Note_Velocity_94",
96 : "Note_Velocity_95",
97 : "Note_Velocity_96",
98 : "Note_Velocity_97",
99 : "Note_Velocity_98",
100: "Note_Velocity_99",
101: "Note_Velocity_100",
102: "Note_Velocity_101",
103: "Note_Velocity_102",
104: "Note_Velocity_103",
105: "Note_Velocity_104",
106: "Note_Velocity_105",
107: "Note_Velocity_106",
108: "Note_Velocity_107",
109: "Note_Velocity_108",
110: "Note_Velocity_109",
111: "Note_Velocity_110",
112: "Note_Velocity_111",
113: "Note_Velocity_112",
114: "Note_Velocity_113",
115: "Note_Velocity_114",
116: "Note_Velocity_115",
117: "Note_Velocity_116",
118: "Note_Velocity_117",
119: "Note_Velocity_118",
120: "Note_Velocity_119",
121: "Note_Velocity_120",
122: "Note_Velocity_121",
123: "Note_Velocity_122",
124: "Note_Velocity_123",
125: "Note_Velocity_124",
126: "Note_Velocity_125",
127: "Note_Velocity_126",
128: "Note_Velocity_127",
129: "Note_Velocity_128",
130: "Note_Velocity_129",
131: "Note_Velocity_130",
132: "Note_Velocity_131",
133: "Note_Velocity_132",
134: "Note_Velocity_133",
135: "Note_Velocity_134",
136: "Note_Velocity_135",
137: "Note_Velocity_136",
138: "Note_Velocity_137",
139: "Note_Velocity_138",
140: "Note_Velocity_139",
141: "Note_Velocity_140",
142: "Note_Velocity_141",
143: "Note_Velocity_142",
144: "Note_Velocity_143",
145: "Note_Velocity_144",
146: "Note_Velocity_145",
147: "Note_Velocity_146",
148: "Note_Velocity_147",
149: "Note_Velocity_148",
150: "Note_Velocity_149",
151: "Note_Velocity_150",
152: "Note_Velocity_151",
153: "Note_Velocity_152",
154: "Note_Velocity_153",
155: "Note_Velocity_154",
156: "Note_Velocity_155",
157: "Note_Velocity_156",
158: "Note_Velocity_157",
159: "Note_Velocity_158",
160: "Note_Velocity_159",
161: "Note_Velocity_160",
162: "Note_Velocity_161",
163: "Note_Velocity_162",
164: "Note_Velocity_163",
165: "Note_Velocity_164",
166: "Note_Velocity_165",
167: "Note_Velocity_166",
168: "Note_Velocity_167",
169: "Note_Velocity_168",
170: "Note_Velocity_169",
171: "Note_Velocity_170",
172: "Note_Velocity_171",
173: "Note_Velocity_172",
174: "Note_Velocity_173",
175: "Note_Velocity_174",
176: "Note_Velocity_175",
177: "Note_Velocity_176",
178: "Note_Velocity_177",
179: "Note_Velocity_178",
180: "Note_Velocity_179",
181: "Note_Velocity_180",
182: "Note_Velocity_181",
183: "Note_Velocity_182",
184: "Note_Velocity_183",
185: "Note_Velocity_184",
186: "Note_Velocity_185",
187: "Note_Velocity_186",
188: "Note_Velocity_187",
189: "Note_Velocity_188",
190: "Note_Velocity_189",
191: "Note_Velocity_190",
192: "Note_Velocity_191",
193: "Note_Velocity_192",
194: "Note_Velocity_193",
195: "Note_Velocity_194",
196: "Note_Velocity_195",
197: "Note_Velocity_196",
198: "Note_Velocity_197",
199: "Note_Velocity_198",
200: "Note_Velocity_199",
201: "Note_Velocity_200",
202: "Note_Velocity_201",
203: "Note_Velocity_202",
204: "Note_Velocity_203",
205: "Note_Velocity_204",
206: "Note_Velocity_205",
207: "Note_Velocity_206",
208: "Note_Velocity_207",
209: "Note_Velocity_208",
210: "Note_Velocity_209",
211: "Note_Velocity_210",
212: "Note_Velocity_211",
213: "Note_Velocity_212",
214: "Note_Velocity_213",
215: "Note_Velocity_214",
216: "Note_Velocity_215",
217: "Note_Velocity_216",
218: "Note_Velocity_217",
219: "Note_Velocity_218",
220: "Note_Velocity_219",
221: "Note_Velocity_220",
222: "Note_Velocity_221",
223: "Note_Velocity_222",
224: "Note_Velocity_223",
225: "Note_Velocity_224",
226: "Note_Velocity_225",
227: "Note_Velocity_226",
228: "Note_Velocity_227",
229: "Note_Velocity_228",
230: "Note_Velocity_229",
231: "Note_Velocity_230",
232: "Note_Velocity_231",
233: "Note_Velocity_232",
234: "Note_Velocity_233",
235: "Note_Velocity_234",
236: "Note_Velocity_235",
237: "Note_Velocity_236",
238: "Note_Velocity_237",
239: "Note_Velocity_238",
240: "Note_Velocity_239",
241: "Note_Velocity_240",
242: "Note_Velocity_241",
243: "Note_Velocity_242",
244: "Note_Velocity_243",
245: "Note_Velocity_244",
246: "Note_Velocity_245",
247: "Note_Velocity_246",
248: "Note_Velocity_247",
249: "Note_Velocity_248",
250: "Note_Velocity_249",
251: "Note_Velocity_250",
252: "Note_Velocity_251",
253: "Note_Velocity_252",
254: "Note_Velocity_253",
255: "Note_Velocity_254",
256: "Note_Velocity_255",
257: "Note_Velocity_256",
258: "Note_Velocity_257",
259: "Note_Velocity_258",
260: "Note_Velocity_259",
261: "Note_Velocity_260",
262: "Note_Velocity_261",
263: "Note_Velocity_262",
264: "Note_Velocity_263",
265: "Note_Velocity_264",
266: "Note_Velocity_265",
267: "Note_Velocity_266",
268: "Note_Velocity_267",
269: "Note_Velocity_268",
270: "Note_Velocity_269",
271: "Note_Velocity_270",
272: "Note_Velocity_271",
273: "Note_Velocity_272",
274: "Note_Velocity_273",
275: "Note_Velocity_274",
276: "Note_Velocity_275",
277: "Note_Velocity_276",
278: "Note_Velocity_277",
279: "Note_Velocity_278",
280: "Note_Velocity_279",
281: "Note_Velocity_280",
282: "Note_Velocity_281",
283: "Note_Velocity_282",
284: "Note_Velocity_283",
285: "Note_Velocity_284",
286: "Note_Velocity_285",
287: "Note_Velocity_286",
288: "Note_Velocity_287",
289: "Note_Velocity_288",
290: "Note_Velocity_289",
291: "Note_Velocity_290",
292: "Note_Velocity_291",
293: "Note_Velocity_292",
294: "Note_Velocity_293",
295: "Note_Velocity_294",
296: "Note_Velocity_295",
297: "Note_Velocity_296",
298: "Note_Velocity_297",
299: "Note_Velocity_298",
300: "Note_Velocity_299",
301: "Note_Velocity_300",
302: "Note_Velocity_301",
303: "Note_Velocity_302",
304: "Note_Velocity_303",
305: "Note_Velocity_304",
306: "Note_Velocity_305",
307: "Note_Velocity_306",
308: "Note_Velocity_307",
309: "Note_Velocity_308",
310: "Note_Velocity_309",
311: "Note_Velocity_310",
312: "Note_Velocity_311",
313: "Note_Velocity_312",
314: "Note_Velocity_313",
315: "Note_Velocity_314",
316: "Note_Velocity_315",
317: "Note_Velocity_316",
318: "Note_Velocity_317",
319: "Note_Velocity_318",
320: "Note_Velocity_319",
321: "Note_Velocity_320",
322: "Note_Velocity_321",
323: "Note_Velocity_322",
324: "Note_Velocity_323",
325: "Note_Velocity_324",
326: "Note_Velocity_325",
327: "Note_Velocity_326",
328: "Note_Velocity_327",
329: "Note_Velocity_328",
330: "Note_Velocity_329",
331: "Note_Velocity_330",
332: "Note_Velocity_331",
333: "Note_Velocity_332",
334: "Note_Velocity_333",
335: "Note_Velocity_334",
336: "Note_Velocity_335",
337: "Note_Velocity_336",
338: "Note_Velocity_337",
339: "Note_Velocity_338",
340: "Note_Velocity_339",
341: "Note_Velocity_340",
342: "Note_Velocity_341",
343: "Note_Velocity_342",
344: "Note_Velocity_343",
345: "Note_Velocity_344",
346: "Note_Velocity_345",
347: "Note_Velocity_346",
348: "Note_Velocity_347",
349: "Note_Velocity_348",
350: "Note_Velocity_349",
351: "Note_Velocity_350",
352: "Note_Velocity_351",
353: "Note_Velocity_352",
354: "Note_Velocity_353",
355: "Note_Velocity_354",
356: "Note_Velocity_355",
357: "Note_Velocity_356",
358: "Note_Velocity_357",
359: "Note_Velocity_358",
360: "Note_Velocity_359",
361: "Note_Velocity_360"
},
"boundary" : {
0: 0,
1: "None",
2: "Boundary",
},
"key" : {
0 : 0,
1 : "None",
2 : 'C',
3 : 'C#',
4 : 'D',
5 : 'D#',
6 : 'E',
7 : 'F',
8 : 'F#',
9 : 'G',
10: 'G#',
11: 'A',
12: 'A#',
13: 'B',
14: 'c',
15: 'c#',
16: 'd',
17: 'd#',
18: 'e',
19: 'f',
20: 'f#',
21: 'g',
22: 'g#',
23: 'a',
24: 'a#',
25: 'b',
}
}
init_dictionary = {
"instr_type": {
'None' : 0,
'Drums' : 1,
'Piano' : 2,
'Guitar' : 3,
'Bass' : 4,
'Strings': 5,
},
"key" : {
"None": 0,
'C' : 1,
'C#' : 2,
'D' : 3,
'D#' : 4,
'E' : 5,
'F' : 6,
'F#' : 7,
'G' : 8,
'G#' : 9,
'A' : 10,
'A#' : 11,
'B' : 12,
'c' : 13,
'c#' : 14,
'd' : 15,
'd#' : 16,
'e' : 17,
'f' : 18,
'f#' : 19,
'g' : 20,
'g#' : 21,
'a' : 22,
'a#' : 23,
'b' : 24,
},
"genre" : {
"None" : 0,
'Metal' : 1,
'Country' : 2,
'dance' : 3,
'Electronic': 4,
'Pop' : 5,
'Rock' : 6,
}
}
genre = {
'Metal' : 1,
'Country' : 2,
'dance' : 3,
'Electronic': 4,
'Pop' : 5,
'Rock' : 6,
}
================================================
FILE: src/gen_midi_conditional.py
================================================
import sys
import os
import time
import glob
import numpy as np
import torch
import argparse
sys.path.append("../dataset/")
from numpy2midi_mix import numpy2midi
from model import CMT
def cal_control_error(err_note_number_list, err_beat_number_list):
print("err_note_number_list", err_note_number_list)
print("err_beat_number_list", err_beat_number_list)
print("strength control error", np.mean(err_note_number_list) / 1.83)
print("density control error", np.mean(err_beat_number_list) / 10.90)
def generate():
# path
parser = argparse.ArgumentParser(description="Args for generating background music")
parser.add_argument('-c', '--ckpt', default="../exp/loss_8_params.pt", help="Model checkpoint to be loaded")
parser.add_argument('-f', '--files', required=True, help="Input npz file of a video")
parser.add_argument('-g', '--gpus', help="Id of gpu. Only ONE gpu is needed")
parser.add_argument('-n', '--num_songs', default=1, help="Number of generated songs")
args = parser.parse_args()
num_songs = int(args.num_songs)
if args.gpus is not None:
if not args.gpus.isnumeric():
raise RuntimeError('Only 1 GPU is needed for inference')
os.environ['CUDA_VISIBLE_DEVICES'] = args.gpus
else:
os.environ['CUDA_VISIBLE_DEVICES'] = '0'
path_saved_ckpt = args.ckpt
filelist = glob.glob(args.files)
# change this if using another training set (see the output of decoder_n_class in train.py)
decoder_n_class = [18, 3, 18, 129, 18, 6, 20, 102, 5025]
init_n_token = [7, 1, 6]
# init model
net = torch.nn.DataParallel(CMT(decoder_n_class, init_n_token))
# load model
print('[*] load model from:', path_saved_ckpt)
if torch.cuda.is_available():
net.cuda()
net.eval()
net.load_state_dict(torch.load(path_saved_ckpt))
else:
net.eval()
net.load_state_dict(torch.load(path_saved_ckpt, map_location=torch.device('cpu')))
if len(filelist) == 0:
raise RuntimeError('no npz file in ' + str(filelist))
for file_name in filelist:
# gen
start_time = time.time()
song_time_list = []
words_len_list = []
sidx = 0
while sidx < num_songs:
try:
print("new song")
start_time = time.time()
vlog_npz = np.load(file_name)['input']
vlog_npz = vlog_npz[vlog_npz[:, 2] != 1]
print(vlog_npz)
res, err_note_number_list, err_beat_number_list = net(is_train=False, vlog=vlog_npz, C=0.7)
cal_control_error(err_note_number_list, err_beat_number_list)
numpy2midi(f"{file_name}_{sidx}", res[:, [1, 0, 2, 3, 4, 5, 6]].astype(np.int32))
song_time = time.time() - start_time
word_len = len(res)
print('song time:', song_time)
print('word_len:', word_len)
words_len_list.append(word_len)
song_time_list.append(song_time)
sidx += 1
except KeyboardInterrupt:
raise ValueError(' [x] terminated.')
if __name__ == '__main__':
print("inference")
generate()
================================================
FILE: src/match.py
================================================
import argparse
from tqdm import tqdm
import numpy as np
def _get_density(bar_word):
assert _is_bar_word(bar_word)
if len(bar_word) == 10:
return bar_word[2] - 1
elif len(bar_word) == 6:
return bar_word[1] - 1
else:
raise NotImplementedError
def _get_strength_and_tick(beat_word):
assert _is_beat_word(beat_word)
if len(beat_word) == 10:
return beat_word[6], beat_word[1] - 1
elif len(beat_word) == 6:
return beat_word[2], beat_word[0] - 1
else:
raise NotImplementedError
def _is_bar_word(word):
if len(word) == 10:
return word[0] == 1 and word[1] == 17
elif len(word) == 6:
return word[0] == 17
else:
raise NotImplementedError
def _is_beat_word(word):
if len(word) == 10:
return word[0] == 1 and word[1] > 0 and word[1] < 17
elif len(word) == 6:
return word[0] > 0 and word[0] < 17
else:
raise NotImplementedError
def _get_density_and_strength_from_npz(npz):
l_density = []
l_strength = []
for word in npz:
if _is_bar_word(word):
l_density.append(_get_density(word))
l_strength.append([0] * 16)
elif _is_beat_word(word):
strength, tick = _get_strength_and_tick(word)
l_strength[-1][tick] = strength
return np.asarray(l_density), np.asarray(l_strength)
def cal_matchness(midi_npz, v_density, v_strength):
m_density, m_strength = _get_density_and_strength_from_npz(midi_npz)
n_bar = min(v_density.shape[0], m_density.shape[0])
v_density = v_density[:n_bar]
v_strength = v_strength[:n_bar]
m_density = m_density[:n_bar]
m_strength = m_strength[:n_bar]
m_strength *= (v_strength > 0)
dist = ((v_density - m_density) ** 2).mean() + ((v_strength - m_strength) ** 2).mean()
return 1. / dist
def match_midi(video_npz, all_midi_metadata, all_midi_npz):
res = []
v_density, v_strength = _get_density_and_strength_from_npz(video_npz)
print('Computing matching scores:')
for i, midi_npz in enumerate(tqdm(all_midi_npz)):
matchess = cal_matchness(midi_npz, v_density, v_strength)
res.append((all_midi_metadata[i]['id'], matchess))
res.sort(key=lambda x: x[1], reverse=True)
print('IDs and matching scores of the 5 most matching music pieces:')
for i in range(5):
print(res[i][0], res[i][1])
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('video', default='../inference/wzk.npz', help='Video npz path')
parser.add_argument('music_lib', default='../dataset/lpd_5_prcem_mix_v8_10000.npz', help='Music npz path')
args = parser.parse_args()
video_npz = np.load(args.video, allow_pickle=True)['input']
tmp = np.load(args.music_lib, allow_pickle=True)
all_midi_metadata = tmp['metadata']
all_midi_npz = tmp['x']
del tmp
match_midi(video_npz, all_midi_metadata, all_midi_npz)
================================================
FILE: src/midi2mp3.py
================================================
import note_seq
from pretty_midi import PrettyMIDI
import midi2audio
import argparse
SAMPLE_RATE = 16000
SF2_PATH = '../SGM-v2.01-Sal-Guit-Bass-V1.3.sf2'
def midi_to_mp3(midi_path, tempo, mp3_path):
midi_obj = PrettyMIDI(midi_path)
# convert tempo
midi_length = midi_obj.get_end_time()
midi_obj.adjust_times([0, midi_length], [0, midi_length*120/tempo])
processed_mid = midi_path[:-4] + "_processed.mid"
midi_obj.write(processed_mid)
print("converting into mp3")
fs = midi2audio.FluidSynth(SF2_PATH, sample_rate=SAMPLE_RATE)
fs.midi_to_audio(processed_mid, mp3_path)
print("playing music")
ns = note_seq.midi_io.midi_to_note_sequence(midi_obj)
note_seq.play_sequence(ns, synth=note_seq.fluidsynth, sample_rate=SAMPLE_RATE, sf2_path=SF2_PATH)
note_seq.plot_sequence(ns)
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--input", default="../inference/get_0.mid")
parser.add_argument("--tempo", default="96")
parser.add_argument("--output", default="../inference/get_0.mp3")
args = parser.parse_args()
midi_to_mp3(args.input, float(args.tempo), args.output)
================================================
FILE: src/midi2numpy_mix.py
================================================
import os
import math
import json
import muspy
import numpy as np
from tqdm import tqdm
import argparse
from dictionary_mix import preset_event2word, preset_word2event
#np.random.seed(208)
RESOLUTION = 16 # 每小节16个时间单位
DECODER_MAX_LEN = 10000
# DECODER_MAX_LEN = 3000
DECODER_DIMENSION = {
'type' : 0,
'beat' : 1,
'density' : 2,
'pitch' : 3,
'duration' : 4,
'instr_type': 5,
'strength' : 6,
'i_beat' : 7,
'n_beat' : 8,
'p_beat' : 9,
}
N_DECODER_DIMENSION = len(DECODER_DIMENSION)
KEYS = ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B', 'c', 'c#', 'd', 'd#', 'e', 'f', 'f#', 'g',
'g#', 'a', 'a#', 'b']
class Note:
def __init__(self, muspy_note=None, instr_type=None): # bar starts from 0
if muspy_note is not None and instr_type is not None:
self.time = muspy_note.time
self.bar = self.time // RESOLUTION # 从0开始
self.beat = muspy_note.time % RESOLUTION # 小节内部的第几个beat
self.i_beat = self.bar * RESOLUTION + self.beat # 整首歌的第几个beat
self.pitch = muspy_note.pitch
self.duration = min(RESOLUTION, muspy_note.duration) # TODO: 截断过长的note?
self.instr_type = instr_type
# self.velocity = muspy_note.velocity
self.velocity = 80
def to_decoder_list(self, n_beat: int) -> list:
l = [0] * N_DECODER_DIMENSION
l[DECODER_DIMENSION['type']] = preset_event2word['type']['Note']
l[DECODER_DIMENSION['pitch']] = preset_event2word['pitch']['Note_Pitch_%d' % self.pitch]
l[DECODER_DIMENSION['duration']] = preset_event2word['duration']['Note_Duration_%d' % (self.duration * 120)]
l[DECODER_DIMENSION['instr_type']] = preset_event2word['instr_type'][self.instr_type]
l[DECODER_DIMENSION['i_beat']] = self.i_beat
l[DECODER_DIMENSION['n_beat']] = n_beat
l[DECODER_DIMENSION['p_beat']] = int(self.i_beat / n_beat * 100)
return l
def to_muspy_note(self) -> muspy.Note:
return muspy.Note(time=self.time, pitch=self.pitch, duration=self.duration, velocity=self.velocity)
def from_decoder_array(self, np_array: np.ndarray, bar: int, beat: int) -> muspy.Note:
assert np_array[DECODER_DIMENSION['type']] == preset_event2word['type']['Note']
assert np_array[DECODER_DIMENSION['pitch']] > 0
assert np_array[DECODER_DIMENSION['duration']] > 0
assert np_array[DECODER_DIMENSION['instr_type']] > 0
self.time = bar * RESOLUTION + beat
self.pitch = np_array[DECODER_DIMENSION['pitch']] - 1
self.duration = np_array[DECODER_DIMENSION['duration']] - 1
self.instr_type = preset_word2event['instr_type'][np_array[DECODER_DIMENSION['instr_type']]]
self.velocity = 80
return self.to_muspy_note()
class Bar:
def __init__(self, notes, i_bar):
self.notes = notes
self.n_notes = len(self.notes)
self.i_bar = i_bar
def _get_beat_token(self, note, density, strength, n_beat: int) -> list:
l = [[0] * N_DECODER_DIMENSION]
l[0][DECODER_DIMENSION['type']] = preset_event2word['type']['M']
l[0][DECODER_DIMENSION['beat']] = preset_event2word['beat']['Beat_%d' % note.beat]
l[0][DECODER_DIMENSION['density']] = density + 1
l[0][DECODER_DIMENSION['instr_type']] = preset_event2word['instr_type'][note.instr_type]
l[0][DECODER_DIMENSION['strength']] = strength
l[0][DECODER_DIMENSION['i_beat']] = note.i_beat
l[0][DECODER_DIMENSION['n_beat']] = n_beat
l[0][DECODER_DIMENSION['p_beat']] = int(note.i_beat / n_beat * 100)
return l
def _get_bar_token(self, density, n_beat: int) -> list:
l = [[0] * N_DECODER_DIMENSION]
l[0][DECODER_DIMENSION['type']] = preset_event2word['type']['M']
l[0][DECODER_DIMENSION['beat']] = preset_event2word['beat']['Bar']
l[0][DECODER_DIMENSION['density']] = density + 1
l[0][DECODER_DIMENSION['i_beat']] = self.i_bar * RESOLUTION
l[0][DECODER_DIMENSION['n_beat']] = n_beat
l[0][DECODER_DIMENSION['p_beat']] = int(self.i_bar * RESOLUTION / n_beat * 100)
return l
def to_decoder_list(self, n_beat: int) -> list:
# 逆序构建
n_beats = 0
l = []
if len(self.notes) > 0:
# add the last note token
l = [self.notes[-1].to_decoder_list(n_beat)]
prev_note = self.notes[-1]
n_notes_per_beat = 1
for note in reversed(self.notes[:-1]):
if note.beat != prev_note.beat or note.instr_type != prev_note.instr_type:
# add beat token
l = self._get_beat_token(note=prev_note, density=n_beats, strength=n_notes_per_beat,
n_beat=n_beat) + l
if note.beat != prev_note.beat:
n_beats += 1
n_notes_per_beat = 0
# add note token
l = [note.to_decoder_list(n_beat)] + l
n_notes_per_beat += 1
prev_note = note
# add the first beat token
l = self._get_beat_token(note=prev_note, density=n_beats, strength=n_notes_per_beat, n_beat=n_beat) + l
n_beats += 1
# add bar token
l = self._get_bar_token(density=n_beats, n_beat=n_beat) + l
return l
class MIDI:
def __init__(self, id: str):
self.id = id
self.midi = muspy.read_midi(os.path.join(midi_dir, id + '.mid'))
self.midi.adjust_resolution(target=RESOLUTION // 4)
self.n_beat = self.midi.get_end_time()
self.n_bars = math.ceil((self.n_beat + 1) / RESOLUTION)
muspy_tracks = []
for track in self.midi.tracks: # filter tracks with <=20 notes
if len(track.notes) > 20:
muspy_tracks.append(track)
self.midi.tracks = muspy_tracks
self.instruments = [track.name for track in muspy_tracks]
self.bars = self._get_bars()
def _get_bars(self):
bars = [[] for i in range(self.n_bars)]
for i, track in enumerate(self.midi.tracks):
for j, muspy_note in enumerate(track.notes):
note = Note(muspy_note, track.name)
bars[note.bar].append(note)
new_bars = []
for i in range(len(bars)):
bars[i].sort(key=lambda x: x.time)
new_bars.append(Bar(notes=bars[i], i_bar=i))
return new_bars
def to_decoder_list(self) -> list:
l = []
for bar in self.bars:
l += bar.to_decoder_list(self.n_beat)
l += [[0] * N_DECODER_DIMENSION] # 多一个EOS
mask = [1] * len(l)
return l, mask, len(l)
def midi2numpy(id_list: list):
if not os.path.exists(json_dir):
os.makedirs(json_dir)
decoder = []
decoder_mask = []
metadata_list = []
decoder_len = []
for id in tqdm(id_list):
id_filename = os.path.join(json_dir, id + '.json')
if os.path.exists(id_filename):
with open(id_filename, 'r') as f:
load_dict = json.load(f)
decoder_list = load_dict['decoder_list']
de_mask = load_dict['de_mask']
de_len = load_dict['de_len']
decoder_len.append(de_len)
metadata = load_dict['metadata']
else:
midi = MIDI(id)
decoder_list, de_mask, de_len = midi.to_decoder_list()
decoder_len.append(de_len)
if de_len > DECODER_MAX_LEN:
continue
# Padding to MAX_LEN
decoder_list += [[0] * N_DECODER_DIMENSION] * (DECODER_MAX_LEN - de_len)
de_mask += [0] * (DECODER_MAX_LEN - de_len)
metadata = {'id': id, 'de_len': de_len, 'instruments': midi.instruments, 'genre': "N/A"}
##### genre set to empty
dic = {'decoder_list': decoder_list, 'de_mask': de_mask, 'de_len': de_len, 'metadata': metadata}
with open(id_filename, 'w') as f:
json.dump(dic, f)
decoder.append(decoder_list)
decoder_mask.append(de_mask)
metadata_list.append(metadata)
print('max decoder length: %d' % max(decoder_len))
decoder = np.asarray(decoder, dtype=int)
x = decoder[:, :-1]
y = decoder[:, 1:]
decoder_mask = np.asarray(decoder_mask, dtype=int)
np.savez(npz_filename, x=x, y=y, decoder_mask=decoder_mask, metadata=metadata_list)
print(npz_filename)
print('%d songs' % len(decoder))
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument("--midi_dir", default="../../lpd_5_cleansed_midi/", required=True)
parser.add_argument("--out_name", default="data.npz", required=True)
args = parser.parse_args()
midi_dir = args.midi_dir
npz_filename = os.path.join("../dataset/", args.out_name)
json_dir = os.path.join("../dataset/json/")
id_list = []
for name in os.listdir(midi_dir):
if name.endswith(".mid"):
id_list.append(name[:-4])
elif name.endswith(".midi"):
id_list.append(name[:-5])
midi2numpy(id_list)
================================================
FILE: src/model.py
================================================
import numpy as np
import torch
import torch.cuda
from torch import nn
from utils import Embeddings, BeatPositionalEncoding
from fast_transformers.builders import TransformerEncoderBuilder
from fast_transformers.masking import TriangularCausalMask
D_MODEL = 512
N_LAYER_ENCODER = 12
N_HEAD = 8
ATTN_DECODER = "causal-linear"
################################################################################
# Sampling
################################################################################
# -- temperature -- #
def softmax_with_temperature(logits, temperature):
logits -= np.max(logits)
probs = np.exp(logits / temperature) / np.sum(np.exp(logits / temperature))
return probs
def weighted_sampling(probs):
probs /= (sum(probs) + 1e-10)
sorted_probs = np.sort(probs)[::-1]
sorted_index = np.argsort(probs)[::-1]
try:
word = np.random.choice(sorted_index, size=1, p=sorted_probs)[0]
except:
word = sorted_index[0]
return word
# -- nucleus -- #
def nucleus(probs, p):
probs /= (sum(probs) + 1e-5)
sorted_probs = np.sort(probs)[::-1]
sorted_index = np.argsort(probs)[::-1]
cusum_sorted_probs = np.cumsum(sorted_probs)
after_threshold = cusum_sorted_probs > p
if sum(after_threshold) > 0:
last_index = np.where(after_threshold)[0][0] + 1
candi_index = sorted_index[:last_index]
else:
candi_index = sorted_index[:]
candi_probs = [probs[i] for i in candi_index]
candi_probs /= sum(candi_probs)
word = np.random.choice(candi_index, size=1, p=candi_probs)[0]
return word
def sampling(logit, p=None, t=1.0):
logit = logit.squeeze().cpu().numpy()
probs = softmax_with_temperature(logits=logit, temperature=t)
if p is not None:
cur_word = nucleus(probs, p=p)
else:
cur_word = weighted_sampling(probs)
return cur_word
'''
last dimension of input data | attribute:
0: bar/beat
1: type
2: density
3: pitch
4: duration
5: instr
6: strength onset_density
7: time_encoding
'''
class CMT(nn.Module):
def __init__(self, n_token, init_n_token, is_training=True):
super(CMT, self).__init__()
print("D_MODEL", D_MODEL, " N_LAYER", N_LAYER_ENCODER, " N_HEAD", N_HEAD, "DECODER ATTN", ATTN_DECODER)
# --- params config --- #
self.n_token = n_token
self.d_model = D_MODEL
self.n_layer_encoder = N_LAYER_ENCODER #
# self.n_layer_decoder = N_LAYER_DECODER
self.dropout = 0.1
self.n_head = N_HEAD #
self.d_head = D_MODEL // N_HEAD
self.d_inner = 2048
self.loss_func = nn.CrossEntropyLoss(reduction='none')
# self.emb_sizes = [64, 32, 512, 128, 32]
self.emb_sizes = [64, 32, 64, 512, 128, 32, 64]
self.init_n_token = init_n_token # genre, key, instrument
self.init_emb_sizes = [64, 64, 64]
self.time_encoding_size = 256
# --- modules config --- #
# embeddings
print('>>>>>:', self.n_token)
self.init_emb_genre = Embeddings(self.init_n_token[0], self.init_emb_sizes[0])
self.init_emb_key = Embeddings(self.init_n_token[1], self.init_emb_sizes[1])
self.init_emb_instrument = Embeddings(self.init_n_token[2], self.init_emb_sizes[2])
self.init_in_linear = nn.Linear(int(np.sum(self.init_emb_sizes)), self.d_model)
self.encoder_emb_barbeat = Embeddings(self.n_token[0], self.emb_sizes[0])
self.encoder_emb_type = Embeddings(self.n_token[1], self.emb_sizes[1])
self.encoder_emb_beat_density = Embeddings(self.n_token[2], self.emb_sizes[2])
self.encoder_emb_pitch = Embeddings(self.n_token[3], self.emb_sizes[3])
self.encoder_emb_duration = Embeddings(self.n_token[4], self.emb_sizes[4])
self.encoder_emb_instr = Embeddings(self.n_token[5], self.emb_sizes[5])
self.encoder_emb_onset_density = Embeddings(self.n_token[6], self.emb_sizes[6])
self.encoder_emb_time_encoding = Embeddings(self.n_token[7], self.time_encoding_size)
self.encoder_pos_emb = BeatPositionalEncoding(self.d_model, self.dropout)
# # linear
self.encoder_in_linear = nn.Linear(int(np.sum(self.emb_sizes)), self.d_model)
self.encoder_time_linear = nn.Linear(int(self.time_encoding_size), self.d_model)
self.transformer_encoder = TransformerEncoderBuilder.from_kwargs(
n_layers=self.n_layer_encoder,
n_heads=self.n_head,
query_dimensions=self.d_model // self.n_head,
value_dimensions=self.d_model // self.n_head,
feed_forward_dimensions=2048,
activation='gelu',
dropout=0.1,
attention_type="causal-linear",
).get()
# blend with type
self.project_concat_type = nn.Linear(self.d_model + 32, self.d_model)
# individual output
self.proj_barbeat = nn.Linear(self.d_model, self.n_token[0])
self.proj_type = nn.Linear(self.d_model, self.n_token[1])
self.proj_beat_density = nn.Linear(self.d_model, self.n_token[2])
self.proj_pitch = nn.Linear(self.d_model, self.n_token[3])
self.proj_duration = nn.Linear(self.d_model, self.n_token[4])
self.proj_instr = nn.Linear(self.d_model, self.n_token[5])
self.proj_onset_density = nn.Linear(self.d_model, self.n_token[6])
def compute_loss(self, predict, target, loss_mask):
loss = self.loss_func(predict, target)
loss = loss * loss_mask
loss = torch.sum(loss) / torch.sum(loss_mask)
return loss
def forward_init_token_vis(self, x, memory=None, is_training=True):
emb_genre = self.init_emb_genre(x[..., 0])
emb_key = self.init_emb_key(x[..., 1])
emb_instrument = self.init_emb_instrument(x[..., 2])
return emb_genre, emb_key, emb_instrument
def forward_init_token(self, x, memory=None, is_training=True):
emb_genre = self.init_emb_genre(x[..., 0])
emb_key = self.init_emb_key(x[..., 1])
emb_instrument = self.init_emb_instrument(x[..., 2])
embs = torch.cat(
[
emb_genre,
emb_key,
emb_instrument,
], dim=-1)
encoder_emb_linear = self.init_in_linear(embs)
if is_training:
return encoder_emb_linear
else:
pos_emb = encoder_emb_linear.squeeze(0)
h, memory = self.transformer_encoder(pos_emb, memory=memory)
y_type = self.proj_type(h)
return h, y_type, memory
def forward_hidden(self, x, memory=None, is_training=True, init_token=None):
# linear transformer: b, s, f x.shape=(bs, nf)
# embeddings
emb_barbeat = self.encoder_emb_barbeat(x[..., 0])
emb_type = self.encoder_emb_type(x[..., 1])
emb_beat_density = self.encoder_emb_beat_density(x[..., 2])
emb_pitch = self.encoder_emb_pitch(x[..., 3])
emb_duration = self.encoder_emb_duration(x[..., 4])
emb_instr = self.encoder_emb_instr(x[..., 5])
emb_onset_density = self.encoder_emb_onset_density(x[..., 6])
emb_time_encoding = self.encoder_emb_time_encoding(x[..., 7])
embs = torch.cat(
[
emb_barbeat,
emb_type,
emb_beat_density,
emb_pitch,
emb_duration,
emb_instr,
emb_onset_density
], dim=-1)
encoder_emb_linear = self.encoder_in_linear(embs)
# import ipdb;ipdb.set_trace()
encoder_emb_time_linear = self.encoder_time_linear(emb_time_encoding)
encoder_emb_linear = encoder_emb_linear + encoder_emb_time_linear
encoder_pos_emb = self.encoder_pos_emb(encoder_emb_linear, x[:, :, 8])
if is_training:
assert init_token is not None
init_emb_linear = self.forward_init_token(init_token)
encoder_pos_emb = torch.cat([init_emb_linear, encoder_pos_emb], dim=1)
else:
assert init_token is not None
init_emb_linear = self.forward_init_token(init_token)
encoder_pos_emb = torch.cat([init_emb_linear, encoder_pos_emb], dim=1)
# transformer
if is_training:
attn_mask = TriangularCausalMask(encoder_pos_emb.size(1), device=x.device)
encoder_hidden = self.transformer_encoder(encoder_pos_emb, attn_mask)
# print("forward decoder done")
y_type = self.proj_type(encoder_hidden[:, 7:, :])
return encoder_hidden, y_type
else:
encoder_mask = TriangularCausalMask(encoder_pos_emb.size(1), device=x.device)
h = self.transformer_encoder(encoder_pos_emb, encoder_mask) # y: s x d_model
h = h[:, -1:, :]
h = h.squeeze(0)
y_type = self.proj_type(h)
return h, y_type
def forward_output(self, h, y):
# for training
tf_skip_type = self.encoder_emb_type(y[..., 1])
h = h[:, 7:, :]
# project other
y_concat_type = torch.cat([h, tf_skip_type], dim=-1)
y_ = self.project_concat_type(y_concat_type)
y_barbeat = self.proj_barbeat(y_)
y_beat_density = self.proj_beat_density(y_)
y_pitch = self.proj_pitch(y_)
y_duration = self.proj_duration(y_)
y_instr = self.proj_instr(y_)
y_onset_density = self.proj_onset_density(y_)
# import ipdb;ipdb.set_trace()
return y_barbeat, y_pitch, y_duration, y_instr, y_onset_density, y_beat_density
def forward_output_sampling(self, h, y_type, recurrent=True):
'''
for inference
'''
y_type_logit = y_type[0, :] # dont know wtf
cur_word_type = sampling(y_type_logit, p=0.90)
type_word_t = torch.from_numpy(
np.array([cur_word_type])).long().unsqueeze(0)
if torch.cuda.is_available():
type_word_t = type_word_t.cuda()
tf_skip_type = self.encoder_emb_type(type_word_t).squeeze(0)
# concat
y_concat_type = torch.cat([h, tf_skip_type], dim=-1)
y_ = self.project_concat_type(y_concat_type)
# project other
y_barbeat = self.proj_barbeat(y_)
y_pitch = self.proj_pitch(y_)
y_duration = self.proj_duration(y_)
y_instr = self.proj_instr(y_)
y_onset_density = self.proj_onset_density(y_)
y_beat_density = self.proj_beat_density(y_)
# sampling gen_cond
cur_word_barbeat = sampling(y_barbeat, t=1.2)
cur_word_pitch = sampling(y_pitch, p=0.9)
cur_word_duration = sampling(y_duration, t=2, p=0.9)
cur_word_instr = sampling(y_instr, p=0.90)
cur_word_onset_density = sampling(y_onset_density, p=0.90)
cur_word_beat_density = sampling(y_beat_density, p=0.90)
# collect
next_arr = np.array([
cur_word_barbeat,
cur_word_type,
cur_word_beat_density,
cur_word_pitch,
cur_word_duration,
cur_word_instr,
cur_word_onset_density,
])
return next_arr
def inference_from_scratch(self, **kwargs):
vlog = kwargs['vlog']
C = kwargs['C']
def get_p_beat(cur_bar, cur_beat, n_beat):
all_beat = cur_bar * 16 + cur_beat - 1
p_beat = round(all_beat / n_beat * 100) + 1
return p_beat
dictionary = {'bar': 17}
strength_track_list = [1, 2, 3]
pre_init = np.array([
[5, 0, 0],
[0, 0, 0],
[0, 0, 1],
[0, 0, 2],
[0, 0, 3],
[0, 0, 4],
[0, 0, 5],
])
init = np.array([
[17, 1, vlog[0][1], 0, 0, 0, 0, 1, 0], # bar
])
with torch.no_grad():
final_res = []
h = None
init_t = torch.from_numpy(init).long()
pre_init = torch.from_numpy(pre_init).long().unsqueeze(0)
if torch.cuda.is_available():
pre_init = pre_init.cuda()
init_t = init_t.cuda()
print('------ initiate ------')
for step in range(init.shape[0]):
input_ = init_t[step, :].unsqueeze(0).unsqueeze(0)
print(input_)
final_res.append(init[step, :][None, ...])
h, y_type = self.forward_hidden(input_, is_training=False, init_token=pre_init)
print('------- condition -------')
assert vlog is not None
n_beat = vlog[0][4]
len_vlog = len(vlog)
cur_vlog = 1
cur_track = 0
idx = 0
acc_beat_num = vlog[0][1]
beat_num = {}
acc_note_num = 0
note_num = 0
err_note_number_list = []
err_beat_number_list = []
p_beat = 1
cur_bar = 0
while (True):
# sample others
print(idx, end="\r")
idx += 1
next_arr = self.forward_output_sampling(h, y_type)
if next_arr[1] == 1:
replace = False
if next_arr[1] == 2 and next_arr[5] == 0:
next_arr[5] = 1
print("warning note with instrument 0 detected, replaced by drum###################")
if cur_vlog >= len_vlog:
print("exceed vlog len")
break
vlog_i = vlog[cur_vlog]
if vlog_i[0] == dictionary['bar'] and next_arr[0] == dictionary['bar']:
err_beat_number = np.abs(len(beat_num.keys()) - acc_beat_num)
err_beat_number_list.append(err_beat_number)
flag = (np.random.rand() < C)
print("replace beat density-----", vlog_i, next_arr)
if flag:
next_arr = np.array([17, 1, vlog_i[1], 0, 0, 0, 0])
print("replace beat density-----", next_arr)
beat_num = {}
acc_beat_num = vlog_i[1]
replace = True
cur_vlog += 1
else:
print("replace denied----")
cur_vlog += 1
elif vlog_i[0] < dictionary['bar'] and next_arr[0] >= vlog_i[0]:
err_note_number = np.abs(acc_note_num - note_num)
err_note_number_list.append(err_note_number)
print("replace onset density----", vlog_i, next_arr)
if cur_track == 0:
cur_density = next_arr[2]
flag = (np.random.rand() < C)
if next_arr[0] == dictionary['bar']:
cur_density = 1
next_arr = np.array(
[vlog_i[0], 1, cur_density, 0, 0, strength_track_list[cur_track], vlog_i[2] + 0])
replace = True
acc_note_num = vlog_i[2] + 0
note_num = 0
cur_track += 1
if cur_track >= len(strength_track_list):
cur_track = 0
cur_vlog += 1
if next_arr[1] == 1:
beat_num[next_arr[0]] = 1
elif next_arr[1] == 2 and replace == True:
note_num += 1
if next_arr[0] == dictionary['bar']:
cur_bar += 1
if next_arr[1] == 1:
if next_arr[0] == 17:
cur_beat = 1
else:
cur_beat = next_arr[0]
p_beat = get_p_beat(cur_bar, cur_beat, n_beat)
if p_beat >= 102:
print("exceed max p_beat----")
break
next_arr = np.concatenate([next_arr, [p_beat], [cur_bar * 16 + cur_beat - 1]])
final_res.append(next_arr[None, ...])
print(next_arr)
# forward
input_cur = torch.from_numpy(next_arr).long().unsqueeze(0).unsqueeze(0)
if torch.cuda.is_available():
input_cur = input_cur.cuda()
input_ = torch.cat((input_, input_cur), dim=1)
if replace:
h, y_type = self.forward_hidden(input_, is_training=False, init_token=pre_init)
else:
h, y_type = self.forward_hidden(input_, is_training=False, init_token=pre_init)
if next_arr[1] == 0:
print("EOS predicted")
break
print('\n--------[Done]--------')
final_res = np.concatenate(final_res)
print(final_res.shape)
return final_res, err_note_number_list, err_beat_number_list
def train_forward(self, **kwargs):
x = kwargs['x']
target = kwargs['target']
loss_mask = kwargs['loss_mask']
init_token = kwargs['init_token']
h, y_type = self.forward_hidden(x, memory=None, is_training=True, init_token=init_token)
y_barbeat, y_pitch, y_duration, y_instr, y_onset_density, y_beat_density = self.forward_output(h, target)
# reshape (b, s, f) -> (b, f, s)
y_barbeat = y_barbeat[:, ...].permute(0, 2, 1)
y_type = y_type[:, ...].permute(0, 2, 1)
y_pitch = y_pitch[:, ...].permute(0, 2, 1)
y_duration = y_duration[:, ...].permute(0, 2, 1)
y_instr = y_instr[:, ...].permute(0, 2, 1)
y_onset_density = y_onset_density[:, ...].permute(0, 2, 1)
y_beat_density = y_beat_density[:, ...].permute(0, 2, 1)
# loss
loss_barbeat = self.compute_loss(
y_barbeat, target[..., 0], loss_mask)
loss_type = self.compute_loss(
y_type, target[..., 1], loss_mask)
loss_beat_density = self.compute_loss(
y_beat_density, target[..., 2], loss_mask)
loss_pitch = self.compute_loss(
y_pitch, target[..., 3], loss_mask)
loss_duration = self.compute_loss(
y_duration, target[..., 4], loss_mask)
loss_instr = self.compute_loss(
y_instr, target[..., 5], loss_mask)
loss_onset_density = self.compute_loss(
y_onset_density, target[..., 6], loss_mask)
return loss_barbeat, loss_type, loss_pitch, loss_duration, loss_instr, loss_onset_density, loss_beat_density
def forward(self, **kwargs):
if kwargs['is_train']:
return self.train_forward(**kwargs)
return self.inference_from_scratch(**kwargs)
================================================
FILE: src/numpy2midi_mix.py
================================================
import muspy
import numpy as np
from midi2numpy_mix import Note, DECODER_DIMENSION, RESOLUTION
from dictionary_mix import preset_event2word
INSTRUMENT_PROGRAM = {
'Drums' : 114,
'Piano' : 0, # Acoustic Grand Piano
'Guitar' : 24, # Acoustic Guitar (nylon)
'Bass' : 33, # Electric Bass (finger)
'Strings': 41 # Viola
}
def test_numpy2midi(idx: int) -> muspy.Music:
npz = np.load('lpd_5_ccdepr_mix_v4_10000.npz', allow_pickle=True)
decoder = npz['x'][idx]
name = npz['metadata'][idx]['id']
return numpy2midi(name, decoder)
def numpy2midi(name, decoder: np.ndarray) -> muspy.Music:
muspy_tracks = []
# Decoder
n_bars = -1
beat = 0
track_notes = {instr_type: [] for instr_type in INSTRUMENT_PROGRAM.keys()}
for word in decoder:
w_type = word[DECODER_DIMENSION['type']]
if w_type == preset_event2word['type']['M']:
if word[DECODER_DIMENSION['beat']] == preset_event2word['beat']['Bar']:
n_bars += 1
elif word[DECODER_DIMENSION['beat']] > 0 and word[DECODER_DIMENSION['beat']] < 17:
beat = word[DECODER_DIMENSION['beat']] - 1
elif w_type == preset_event2word['type']['Note']:
note = Note()
muspy_note = note.from_decoder_array(np_array=word, bar=n_bars, beat=beat)
track_notes[note.instr_type].append(muspy_note)
else:
assert w_type == preset_event2word['type']['EOS']
break
for instr_type, muspy_notes in track_notes.items():
muspy_tracks.append(muspy.Track(
program=INSTRUMENT_PROGRAM[instr_type],
is_drum=(instr_type == 'Drums'),
name=instr_type,
notes=muspy_notes
))
muspy_music = muspy.Music(resolution=RESOLUTION // 4, tracks=muspy_tracks)
muspy.write_midi(name + ".mid", muspy_music)
return muspy_music
if __name__ == '__main__':
test_numpy2midi(idx=66)
================================================
FILE: src/pianoroll2midi.py
================================================
"""Author: Shangzhe Di (shangzhe.di@gmail.com)
Convert the pianoroll files (.npz) in the lpd-5/LPD-5-cleansed (https://salu133445.github.io/lakh-pianoroll-dataset/dataset) to midi files.
The pianoroll files are organized as:
lpd_5_cleansed
├── A
│ ├── A
│ │ ├── A
│ │ │ ├── TRAAAGR128F425B14B
│ │ │ │ └── b97c529ab9ef783a849b896816001748.npz
│ │ │ └── TRAAAZF12903CCCF6B
│ │ │ └── 05f21994c71a5f881e64f45c8d706165.npz
│ │ ├── B
│ │ │ └── TRAABXH128F42955D6
│ │ │ └── 04266ac849c1d3814dc03bbf61511b33.npz
...
The converted midi files will be organized as:
lpd_5_cleansed_midi/
├── TRDNFHP128F9324C47.mid
├── TRHIZHZ128F4295EFA.mid
├── TRRRAJP128E0793859.mid
├── TRRREEC128F9336C97.mid
├── TRRREEO128F933C62F.mid
├── TRRRERM128F429B7A0.mid
├── TRRRGET128F426655D.mid
...
"""
import pypianoroll
import argparse
import os
import os.path as osp
from tqdm import tqdm
def process_dataset(in_dir, out_dir):
if not os.path.exists(out_dir):
os.makedirs(out_dir)
in_filename_list, out_filename_list = [], []
for main_dir, sub_dir, filename_list in os.walk(in_dir):
for filename in filename_list:
if '.npz' not in filename:
continue
in_filename_list.append(osp.join(main_dir, filename))
track_name = main_dir.split("/")[-1]
out_filename_list.append(osp.join(out_dir, track_name + '.mid'))
for i, in_filename in enumerate(tqdm(in_filename_list)):
convert_midi(in_filename, out_filename_list[i])
def convert_midi(in_filename, out_filename):
pianoroll = pypianoroll.load(in_filename)
pypianoroll.write(out_filename, pianoroll)
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument("--in_dir", help='the directory of the LPD dataset', default='./lpd_5_cleansed/')
parser.add_argument("--out_dir", help='the directory of the output midi files', default='./lpd_5_cleansed_midi/')
args = parser.parse_args()
process_dataset(args.in_dir, args.out_dir)
================================================
FILE: src/train.py
================================================
import sys
import datetime
import argparse
import os
import time
import numpy as np
import torch
from torch import optim
from torch.nn.utils import clip_grad_norm_
sys.path.append(".")
import utils
from utils import log, Saver, network_paras
from model import CMT
def train_dp():
parser = argparse.ArgumentParser(description="Args for training CMT")
parser.add_argument('-n', '--name', default="debug",
help="Name of the experiment, also the log file and checkpoint directory. If 'debug', checkpoints won't be saved")
parser.add_argument('-l', '--lr', default=0.0001, help="Initial learning rate")
parser.add_argument('-b', '--batch_size', default=6, help="Batch size")
parser.add_argument('-p', '--path', help="If set, load model from the given path")
parser.add_argument('-e', '--epochs', default=200, help="Num of epochs")
parser.add_argument('-t', '--train_data', default='../dataset/lpd_5_prcem_mix_v8_10000.npz',
help="Path of the training data (.npz file)")
parser.add_argument('-g', '--gpus', type=int, nargs='+', help="Ids of gpu")
args = parser.parse_args()
if args.gpus is None:
os.environ['CUDA_VISIBLE_DEVICES'] = ",".join([str(g) for g in list(range(torch.cuda.device_count()))])
else:
os.environ['CUDA_VISIBLE_DEVICES'] = ",".join([str(g) for g in args.gpus])
path_train_data = args.train_data
init_lr = float(args.lr)
batch_size = int(args.batch_size)
DEBUG = args.name == "debug"
params = {
"DECAY_EPOCH": [],
"DECAY_RATIO": 0.1,
}
log("name:", args.name)
log("args", args)
if DEBUG:
log("DEBUG MODE checkpoints will not be saved")
else:
utils.flog = open("../logs/" + args.name + ".log", "w")
# hyper params
n_epoch = args.epochs
max_grad_norm = 3
# config
train_data = np.load(path_train_data, allow_pickle=True)
train_x = train_data['x'][:, :, [1, 0, 2, 3, 4, 5, 6, 9, 7]]
train_y = train_data['y'][:, :, [1, 0, 2, 3, 4, 5, 6, 9, 7]]
train_mask = train_data['decoder_mask'][:, :9999]
metadata = train_data['metadata']
for i, m in enumerate(metadata):
total = m["de_len"] - 1
train_x[i, :total, 7] = train_x[i, :total, 7] + 1
init_token = np.tile(np.array([
[5, 0, 0],
[0, 0, 0],
[0, 0, 1],
[0, 0, 2],
[0, 0, 3],
[0, 0, 4],
[0, 0, 5],
]), (train_x.shape[0], 1, 1))
num_batch = len(train_x) // batch_size
# create saver
saver_agent = Saver(exp_dir="../exp/" + args.name, debug=DEBUG)
decoder_n_class = np.max(train_x, axis=(0, 1)) + 1
init_n_class = [7, 1, 6]
# decoder_n_class = [18, 3, 18, 129, 18, 6, 20, 102, 5025]
# init_n_class = [7, 1, 6]
# log
log('num of encoder classes:', decoder_n_class, init_n_class)
# init
net = torch.nn.DataParallel(CMT(decoder_n_class, init_n_class))
if torch.cuda.is_available():
net.cuda()
DEVICE_COUNT = torch.cuda.device_count()
log("DEVICE COUNT:", DEVICE_COUNT)
log("VISIBLE: " + os.environ["CUDA_VISIBLE_DEVICES"])
net.train()
n_parameters = network_paras(net)
log('n_parameters: {:,}'.format(n_parameters))
saver_agent.add_summary_msg(
' > params amount: {:,d}'.format(n_parameters))
if args.path is not None:
print('[*] load model from:', args.path)
net.load_state_dict(torch.load(args.path))
# optimizers
optimizer = optim.Adam(net.parameters(), lr=init_lr)
log(' train_data:', path_train_data.split("/")[-2])
log(' batch_size:', batch_size)
log(' num_batch:', num_batch)
log(' train_x:', train_x.shape)
log(' train_y:', train_y.shape)
log(' train_mask:', train_mask.shape)
log(' lr_init:', init_lr)
for k, v in params.items():
log(f' {k}: {v}')
# run
start_time = time.time()
for epoch in range(n_epoch):
acc_loss = 0
acc_losses = np.zeros(7)
if epoch in params['DECAY_EPOCH']:
log('LR decay by ratio', params['DECAY_RATIO'])
for p in optimizer.param_groups:
p['lr'] *= params['DECAY_RATIO']
for bidx in range(num_batch): # num_batch
saver_agent.global_step_increment()
# index
bidx_st = batch_size * bidx
bidx_ed = batch_size * (bidx + 1)
# unpack batch data
batch_x = train_x[bidx_st:bidx_ed]
batch_y = train_y[bidx_st:bidx_ed]
batch_mask = train_mask[bidx_st:bidx_ed]
batch_init = init_token[bidx_st:bidx_ed]
# to tensor
batch_x = torch.from_numpy(batch_x).long()
batch_y = torch.from_numpy(batch_y).long()
batch_mask = torch.from_numpy(batch_mask).float()
batch_init = torch.from_numpy(batch_init).long()
if torch.cuda.is_available():
batch_x = batch_x.cuda()
batch_y = batch_y.cuda()
batch_mask = batch_mask.cuda()
batch_init = batch_init.cuda()
# run
losses = net(is_train=True, x=batch_x, target=batch_y, loss_mask=batch_mask, init_token=batch_init)
losses = [l.sum() for l in losses]
loss = (losses[0] + losses[1] + losses[2] + losses[3] + losses[4] + losses[5] + losses[6]) / 7
# Update
net.zero_grad()
loss.backward()
if max_grad_norm is not None:
clip_grad_norm_(net.parameters(), max_grad_norm)
optimizer.step()
# print
sys.stdout.write(
'{}/{} | Loss: {:.3f} | barbeat {:.3f}, type {:.3f}, pitch {:.3f}, duration {:.3f}, instr {:.3f}, strength {:.3f}, density {:.3f}\r'.format(
bidx, num_batch, float(loss), losses[0], losses[1], losses[2], losses[3], losses[4], losses[5],
losses[6]))
sys.stdout.flush()
# acc
acc_losses += np.array([l.item() for l in losses])
acc_loss += loss.item()
# log
saver_agent.add_summary('batch loss', loss.item())
# epoch loss
runtime = time.time() - start_time
epoch_loss = acc_loss / num_batch
acc_losses = acc_losses / num_batch
log('-' * 80)
log(time.ctime() + ' epoch: {}/{} | Loss: {:.3f} | time: {}'.format(
epoch, n_epoch, epoch_loss, str(datetime.timedelta(seconds=runtime))))
each_loss_str = 'barbeat {:.3f}, type {:.3f}, pitch {:.3f}, duration {:.3f}, instr {:.3f}, strength {:.3f}, density {:.3f}\r'.format(
acc_losses[0], acc_losses[1], acc_losses[2], acc_losses[3], acc_losses[4], acc_losses[5], acc_losses[6])
log('each loss > ' + each_loss_str)
saver_agent.add_summary('epoch loss', epoch_loss)
saver_agent.add_summary('epoch each loss', each_loss_str)
# save model, with policy
loss = epoch_loss
if 0.2 < loss:
fn = int(loss * 10) * 10
saver_agent.save_model(net, name='loss_' + str(fn))
elif 0.001 < loss <= 0.20:
fn = int(loss * 100)
saver_agent.save_model(net, name='loss_' + str(fn))
elif loss <= 0.001:
log('Finished')
return
# else:
# saver_agent.save_model(net, name='loss_high')
if __name__ == '__main__':
train_dp()
================================================
FILE: src/utils.py
================================================
import math
import numpy as np
import torch
import torch.nn as nn
import os
import time
import collections
import matplotlib.pyplot as plt
import logging
flog = None
################################################################################
# Sampling
################################################################################
# -- temperature -- #
def softmax_with_temperature(logits, temperature):
probs = np.exp(logits / temperature) / np.sum(np.exp(logits / temperature))
return probs
def weighted_sampling(probs):
probs /= sum(probs)
sorted_probs = np.sort(probs)[::-1]
sorted_index = np.argsort(probs)[::-1]
word = np.random.choice(sorted_index, size=1, p=sorted_probs)[0]
return word
# -- nucleus -- #
def nucleus(probs, p):
probs /= (sum(probs) + 1e-5)
sorted_probs = np.sort(probs)[::-1]
sorted_index = np.argsort(probs)[::-1]
cusum_sorted_probs = np.cumsum(sorted_probs)
after_threshold = cusum_sorted_probs > p
if sum(after_threshold) > 0:
last_index = np.where(after_threshold)[0][0] + 1
candi_index = sorted_index[:last_index]
else:
candi_index = sorted_index[:]
candi_probs = [probs[i] for i in candi_index]
candi_probs /= sum(candi_probs)
word = np.random.choice(candi_index, size=1, p=candi_probs)[0]
return word
def sampling(logit, p=None, t=1.0):
logit = logit.squeeze().cpu().numpy()
probs = softmax_with_temperature(logits=logit, temperature=t)
if p is not None:
cur_word = nucleus(probs, p=p)
else:
cur_word = weighted_sampling(probs)
return cur_word
################################################################################
# Model
################################################################################
def network_paras(model):
# compute only trainable params
model_parameters = filter(lambda p: p.requires_grad, model.parameters())
params = sum([np.prod(p.size()) for p in model_parameters])
return params
class Embeddings(nn.Module):
def __init__(self, n_token, d_model):
super(Embeddings, self).__init__()
self.lut = nn.Embedding(n_token, d_model)
self.d_model = d_model
def forward(self, x):
return self.lut(x) * math.sqrt(self.d_model)
class PositionalEncoding(nn.Module):
def __init__(self, d_model, dropout=0.1, max_len=20000):
super(PositionalEncoding, self).__init__()
self.dropout = nn.Dropout(p=dropout)
pe = torch.zeros(max_len, d_model)
position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)
div_term = torch.exp(torch.arange(0, d_model, 2).float() * (-math.log(10000.0) / d_model))
pe[:, 0::2] = torch.sin(position * div_term)
pe[:, 1::2] = torch.cos(position * div_term)
pe = pe.unsqueeze(0)
self.register_buffer('pe', pe)
def forward(self, x):
# print(x.shape, self.pe[:, :x.size(1), :].shape)
x = x + self.pe[:, :x.size(1), :]
return self.dropout(x)
class BeatPositionalEncoding(nn.Module):
def __init__(self, d_model, dropout=0.1, max_len=20000):
super(BeatPositionalEncoding, self).__init__()
self.dropout = nn.Dropout(p=dropout)
pe = torch.zeros(max_len, d_model)
position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)
div_term = torch.exp(torch.arange(0, d_model, 2).float() * (-math.log(10000.0) / d_model))
pe[:, 0::2] = torch.sin(position * div_term)
pe[:, 1::2] = torch.cos(position * div_term)
pe = pe.unsqueeze(0)
self.register_buffer('pe', pe)
def forward(self, x, index):
x = x + self.pe[:, index, :]
return self.dropout(x)[0]
class Saver(object):
def __init__(
self,
exp_dir,
mode='w',
debug=False):
self.exp_dir = exp_dir
self.init_time = time.time()
self.global_step = 0
self.debug = debug
# makedirs
os.makedirs(exp_dir, exist_ok=True)
# logging config
path_logger = os.path.join(exp_dir, 'log.txt')
logging.basicConfig(
level=logging.DEBUG,
format='%(message)s',
filename=path_logger,
filemode=mode)
self.logger = logging.getLogger('training monitor')
def add_summary_msg(self, msg):
if self.debug:
return
self.logger.debug(msg)
def add_summary(
self,
key,
val,
step=None,
cur_time=None):
if self.debug:
return
if cur_time is None:
cur_time = time.time() - self.init_time
if step is None:
step = self.global_step
# write msg (key, val, step, time)
if isinstance(val, float):
msg_str = '{:10s} | {:.10f} | {:10d} | {}'.format(
key,
val,
step,
cur_time
)
else:
msg_str = '{:10s} | {} | {:10d} | {}'.format(
key,
val,
step,
cur_time
)
self.logger.debug(msg_str)
def save_model(
self,
model,
optimizer=None,
outdir=None,
name='model'):
if self.debug:
return
if outdir is None:
outdir = self.exp_dir
print(' [*] saving model to {}, name: {}'.format(outdir, name))
# torch.save(model, os.path.join(outdir, name+'.pt'))
torch.save(model.state_dict(), os.path.join(outdir, name + '_params.pt'))
if optimizer is not None:
torch.save(optimizer.state_dict(), os.path.join(outdir, name + '_opt.pt'))
def load_model(
self,
path_exp,
device='cpu',
name='model.pt'):
path_pt = os.path.join(path_exp, name)
print(' [*] restoring model from', path_pt)
model = torch.load(path_pt, map_location=torch.device(device))
return model
def global_step_increment(self):
self.global_step += 1
def make_loss_report(
path_log,
path_figure='loss.png',
dpi=100):
# load logfile
monitor_vals = collections.defaultdict(list)
with open(path_log, 'r') as f:
for line in f:
try:
line = line.strip()
key, val, step, acc_time = line.split(' | ')
monitor_vals[key].append((float(val), int(step), acc_time))
except:
continue
# collect
step_train = [item[1] for item in monitor_vals['train loss']]
vals_train = [item[0] for item in monitor_vals['train loss']]
step_valid = [item[1] for item in monitor_vals['valid loss']]
vals_valid = [item[0] for item in monitor_vals['valid loss']]
x_min = step_valid[np.argmin(vals_valid)]
y_min = min(vals_valid)
# plot
fig = plt.figure(dpi=dpi)
plt.title('training process')
plt.plot(step_train, vals_train, label='train')
plt.plot(step_valid, vals_valid, label='valid')
plt.yscale('log')
plt.plot([x_min], [y_min], 'ro')
plt.legend(loc='upper right')
plt.tight_layout()
plt.savefig(path_figure)
def log(*args, **kwargs):
print(*args, **kwargs)
if flog is not None:
print(*args, file=flog, flush=True)
================================================
FILE: src/video2npz/dictionary_mix.py
================================================
preset_event2word = {
"tempo" : {
0 : 0,
"CONTI" : 1,
"Tempo_0" : 2,
"Tempo_1" : 3,
"Tempo_2" : 4,
"Tempo_3" : 5,
"Tempo_4" : 6,
"Tempo_5" : 7,
"Tempo_6" : 8,
"Tempo_7" : 9,
"Tempo_8" : 10,
"Tempo_9" : 11,
"Tempo_10" : 12,
"Tempo_11" : 13,
"Tempo_12" : 14,
"Tempo_13" : 15,
"Tempo_14" : 16,
"Tempo_15" : 17,
"Tempo_16" : 18,
"Tempo_17" : 19,
"Tempo_18" : 20,
"Tempo_19" : 21,
"Tempo_20" : 22,
"Tempo_21" : 23,
"Tempo_22" : 24,
"Tempo_23" : 25,
"Tempo_24" : 26,
"Tempo_25" : 27,
"Tempo_26" : 28,
"Tempo_27" : 29,
"Tempo_28" : 30,
"Tempo_29" : 31,
"Tempo_30" : 32,
"Tempo_31" : 33,
"Tempo_32" : 34,
"Tempo_33" : 35,
"Tempo_34" : 36,
"Tempo_35" : 37,
"Tempo_36" : 38,
"Tempo_37" : 39,
"Tempo_38" : 40,
"Tempo_39" : 41,
"Tempo_40" : 42,
"Tempo_41" : 43,
"Tempo_42" : 44,
"Tempo_43" : 45,
"Tempo_44" : 46,
"Tempo_45" : 47,
"Tempo_46" : 48,
"Tempo_47" : 49,
"Tempo_48" : 50,
"Tempo_49" : 51,
"Tempo_50" : 52,
"Tempo_51" : 53,
"Tempo_52" : 54,
"Tempo_53" : 55,
"Tempo_54" : 56,
"Tempo_55" : 57,
"Tempo_56" : 58,
"Tempo_57" : 59,
"Tempo_58" : 60,
"Tempo_59" : 61,
"Tempo_60" : 62,
"Tempo_61" : 63,
"Tempo_62" : 64,
"Tempo_63" : 65,
"Tempo_64" : 66,
"Tempo_65" : 67,
"Tempo_66" : 68,
"Tempo_67" : 69,
"Tempo_68" : 70,
"Tempo_69" : 71,
"Tempo_70" : 72,
"Tempo_71" : 73,
"Tempo_72" : 74,
"Tempo_73" : 75,
"Tempo_74" : 76,
"Tempo_75" : 77,
"Tempo_76" : 78,
"Tempo_77" : 79,
"Tempo_78" : 80,
"Tempo_79" : 81,
"Tempo_80" : 82,
"Tempo_81" : 83,
"Tempo_82" : 84,
"Tempo_83" : 85,
"Tempo_84" : 86,
"Tempo_85" : 87,
"Tempo_86" : 88,
"Tempo_87" : 89,
"Tempo_88" : 90,
"Tempo_89" : 91,
"Tempo_90" : 92,
"Tempo_91" : 93,
"Tempo_92" : 94,
"Tempo_93" : 95,
"Tempo_94" : 96,
"Tempo_95" : 97,
"Tempo_96" : 98,
"Tempo_97" : 99,
"Tempo_98" : 100,
"Tempo_99" : 101,
"Tempo_100": 102,
"Tempo_101": 103,
"Tempo_102": 104,
"Tempo_103": 105,
"Tempo_104": 106,
"Tempo_105": 107,
"Tempo_106": 108,
"Tempo_107": 109,
"Tempo_108": 110,
"Tempo_109": 111,
"Tempo_110": 112,
"Tempo_111": 113,
"Tempo_112": 114,
"Tempo_113": 115,
"Tempo_114": 116,
"Tempo_115": 117,
"Tempo_116": 118,
"Tempo_117": 119,
"Tempo_118": 120,
"Tempo_119": 121,
"Tempo_120": 122,
"Tempo_121": 123,
"Tempo_122": 124,
"Tempo_123": 125,
"Tempo_124": 126,
"Tempo_125": 127,
"Tempo_126": 128,
"Tempo_127": 129,
"Tempo_128": 130,
"Tempo_129": 131,
"Tempo_130": 132,
"Tempo_131": 133,
"Tempo_132": 134,
"Tempo_133": 135,
"Tempo_134": 136,
"Tempo_135": 137,
"Tempo_136": 138,
"Tempo_137": 139,
"Tempo_138": 140,
"Tempo_139": 141,
"Tempo_140": 142,
"Tempo_141": 143,
"Tempo_142": 144,
"Tempo_143": 145,
"Tempo_144": 146,
"Tempo_145": 147,
"Tempo_146": 148,
"Tempo_147": 149,
"Tempo_148": 150,
"Tempo_149": 151,
"Tempo_150": 152,
"Tempo_151": 153,
"Tempo_152": 154,
"Tempo_153": 155,
"Tempo_154": 156,
"Tempo_155": 157,
"Tempo_156": 158,
"Tempo_157": 159,
"Tempo_158": 160,
"Tempo_159": 161,
"Tempo_160": 162,
"Tempo_161": 163,
"Tempo_162": 164,
"Tempo_163": 165,
"Tempo_164": 166,
"Tempo_165": 167,
"Tempo_166": 168,
"Tempo_167": 169,
"Tempo_168": 170,
"Tempo_169": 171,
"Tempo_170": 172,
"Tempo_171": 173,
"Tempo_172": 174,
"Tempo_173": 175,
"Tempo_174": 176,
"Tempo_175": 177,
"Tempo_176": 178,
"Tempo_177": 179,
"Tempo_178": 180,
"Tempo_179": 181,
"Tempo_180": 182,
"Tempo_181": 183,
"Tempo_182": 184,
"Tempo_183": 185,
"Tempo_184": 186,
"Tempo_185": 187,
"Tempo_186": 188,
"Tempo_187": 189,
"Tempo_188": 190,
"Tempo_189": 191,
"Tempo_190": 192,
"Tempo_191": 193,
"Tempo_192": 194,
"Tempo_193": 195,
"Tempo_194": 196,
"Tempo_195": 197,
"Tempo_196": 198,
"Tempo_197": 199,
"Tempo_198": 200,
"Tempo_199": 201,
"Tempo_200": 202,
"Tempo_201": 203,
"Tempo_202": 204,
"Tempo_203": 205,
"Tempo_204": 206,
"Tempo_205": 207,
"Tempo_206": 208,
"Tempo_207": 209,
"Tempo_208": 210,
"Tempo_209": 211,
"Tempo_210": 212,
"Tempo_211": 213,
"Tempo_212": 214,
"Tempo_213": 215,
"Tempo_214": 216,
"Tempo_215": 217,
"Tempo_216": 218,
"Tempo_217": 219,
"Tempo_218": 220,
"Tempo_219": 221,
"Tempo_220": 222,
"Tempo_221": 223,
"Tempo_222": 224,
"Tempo_223": 225,
"Tempo_224": 226,
"Tempo_225": 227,
"Tempo_226": 228,
"Tempo_227": 229,
"Tempo_228": 230,
"Tempo_229": 231,
"Tempo_230": 232,
"Tempo_231": 233,
"Tempo_232": 234,
"Tempo_233": 235,
"Tempo_234": 236,
"Tempo_235": 237,
"Tempo_236": 238,
"Tempo_237": 239,
"Tempo_238": 240,
"Tempo_239": 241,
"Tempo_240": 242,
"Tempo_241": 243,
"Tempo_242": 244,
"Tempo_243": 245,
"Tempo_244": 246,
"Tempo_245": 247,
"Tempo_246": 248,
"Tempo_247": 249,
"Tempo_248": 250,
"Tempo_249": 251,
"Tempo_250": 252,
"Tempo_251": 253,
"Tempo_252": 254,
"Tempo_253": 255,
"Tempo_254": 256,
"Tempo_255": 257,
"Tempo_256": 258,
"Tempo_257": 259,
"Tempo_258": 260,
"Tempo_259": 261,
"Tempo_260": 262,
"Tempo_261": 263,
"Tempo_262": 264,
"Tempo_263": 265,
"Tempo_264": 266,
"Tempo_265": 267,
"Tempo_266": 268,
"Tempo_267": 269,
"Tempo_268": 270,
"Tempo_269": 271,
"Tempo_270": 272,
"Tempo_271": 273,
"Tempo_272": 274,
"Tempo_273": 275,
"Tempo_274": 276,
"Tempo_275": 277,
"Tempo_276": 278,
"Tempo_277": 279,
"Tempo_278": 280,
"Tempo_279": 281,
"Tempo_280": 282,
"Tempo_281": 283,
"Tempo_282": 284,
"Tempo_283": 285,
"Tempo_284": 286,
"Tempo_285": 287,
"Tempo_286": 288,
"Tempo_287": 289,
"Tempo_288": 290,
"Tempo_289": 291,
"Tempo_290": 292,
"Tempo_291": 293,
"Tempo_292": 294,
"Tempo_293": 295,
"Tempo_294": 296,
"Tempo_295": 297,
"Tempo_296": 298,
"Tempo_297": 299,
"Tempo_298": 300,
"Tempo_299": 301,
"Tempo_300": 302,
"Tempo_301": 303,
"Tempo_302": 304,
"Tempo_303": 305,
"Tempo_304": 306,
"Tempo_305": 307,
"Tempo_306": 308,
"Tempo_307": 309,
"Tempo_308": 310,
"Tempo_309": 311,
"Tempo_310": 312,
"Tempo_311": 313,
"Tempo_312": 314,
"Tempo_313": 315,
"Tempo_314": 316,
"Tempo_315": 317,
"Tempo_316": 318,
"Tempo_317": 319,
"Tempo_318": 320,
"Tempo_319": 321,
"Tempo_320": 322,
"Tempo_321": 323,
"Tempo_322": 324,
"Tempo_323": 325,
"Tempo_324": 326,
"Tempo_325": 327,
"Tempo_326": 328,
"Tempo_327": 329,
"Tempo_328": 330,
"Tempo_329": 331,
"Tempo_330": 332,
"Tempo_331": 333,
"Tempo_332": 334,
"Tempo_333": 335,
"Tempo_334": 336,
"Tempo_335": 337,
"Tempo_336": 338,
"Tempo_337": 339,
"Tempo_338": 340,
"Tempo_339": 341,
"Tempo_340": 342,
"Tempo_341": 343,
"Tempo_342": 344,
"Tempo_343": 345,
"Tempo_344": 346,
"Tempo_345": 347,
"Tempo_346": 348,
"Tempo_347": 349,
"Tempo_348": 350,
"Tempo_349": 351,
"Tempo_350": 352,
"Tempo_351": 353,
"Tempo_352": 354,
"Tempo_353": 355,
"Tempo_354": 356,
"Tempo_355": 357,
"Tempo_356": 358,
"Tempo_357": 359,
"Tempo_358": 360,
"Tempo_359": 361,
"Tempo_360": 362
},
"chord" : {
0 : 0,
"CONTI": 1
},
"beat" : {
0 : 0,
"Beat_0" : 1,
"Beat_1" : 2,
"Beat_2" : 3,
"Beat_3" : 4,
"Beat_4" : 5,
"Beat_5" : 6,
"Beat_6" : 7,
"Beat_7" : 8,
"Beat_8" : 9,
"Beat_9" : 10,
"Beat_10": 11,
"Beat_11": 12,
"Beat_12": 13,
"Beat_13": 14,
"Beat_14": 15,
"Beat_15": 16,
"Bar" : 17,
},
"type" : {
"EOS" : 0,
"M" : 1,
"Note" : 2,
'Global': 3,
},
"instr_type": {
'None' : 0,
'Drums' : 1,
'Piano' : 2,
'Guitar' : 3,
'Bass' : 4,
'Strings': 5,
},
"key/genre" : {
"None" : 0,
'C' : 5,
'C#' : 6,
'D' : 7,
'D#' : 8,
'E' : 9,
'F' : 10,
'F#' : 11,
'G' : 12,
'G#' : 13,
'A' : 14,
'A#' : 15,
'B' : 16,
'c' : 17,
'c#' : 18,
'd' : 19,
'd#' : 20,
'e' : 21,
'f' : 22,
'f#' : 23,
'g' : 24,
'g#' : 25,
'a' : 26,
'a#' : 27,
'b' : 28,
'classic' : 29,
'country' : 30,
'dance' : 31,
'electronic': 32,
'pop' : 33,
'rock' : 34,
# 'happy': 29,
# 'sad': 30,
# 'Rock': 29,
# 'Rap': 30,
# 'Latin': 31,
# 'Jazz': 32,
# 'Electronic': 33,
# 'Punk': 34,
# 'Pop': 35,
# 'New Age': 36,
# 'Metal': 37,
# 'RnB': 38,
# 'Country': 39,
# 'Reggae': 40,
# 'Folk': 41,
# 'Blues': 42,
# 'World': 43
},
"pitch" : {
0 : 0,
"Note_Pitch_0" : 1,
"Note_Pitch_1" : 2,
"Note_Pitch_2" : 3,
"Note_Pitch_3" : 4,
"Note_Pitch_4" : 5,
"Note_Pitch_5" : 6,
"Note_Pitch_6" : 7,
"Note_Pitch_7" : 8,
"Note_Pitch_8" : 9,
"Note_Pitch_9" : 10,
"Note_Pitch_10" : 11,
"Note_Pitch_11" : 12,
"Note_Pitch_12" : 13,
"Note_Pitch_13" : 14,
"Note_Pitch_14" : 15,
"Note_Pitch_15" : 16,
"Note_Pitch_16" : 17,
"Note_Pitch_17" : 18,
"Note_Pitch_18" : 19,
"Note_Pitch_19" : 20,
"Note_Pitch_20" : 21,
"Note_Pitch_21" : 22,
"Note_Pitch_22" : 23,
"Note_Pitch_23" : 24,
"Note_Pitch_24" : 25,
"Note_Pitch_25" : 26,
"Note_Pitch_26" : 27,
"Note_Pitch_27" : 28,
"Note_Pitch_28" : 29,
"Note_Pitch_29" : 30,
"Note_Pitch_30" : 31,
"Note_Pitch_31" : 32,
"Note_Pitch_32" : 33,
"Note_Pitch_33" : 34,
"Note_Pitch_34" : 35,
"Note_Pitch_35" : 36,
"Note_Pitch_36" : 37,
"Note_Pitch_37" : 38,
"Note_Pitch_38" : 39,
"Note_Pitch_39" : 40,
"Note_Pitch_40" : 41,
"Note_Pitch_41" : 42,
"Note_Pitch_42" : 43,
"Note_Pitch_43" : 44,
"Note_Pitch_44" : 45,
"Note_Pitch_45" : 46,
"Note_Pitch_46" : 47,
"Note_Pitch_47" : 48,
"Note_Pitch_48" : 49,
"Note_Pitch_49" : 50,
"Note_Pitch_50" : 51,
"Note_Pitch_51" : 52,
"Note_Pitch_52" : 53,
"Note_Pitch_53" : 54,
"Note_Pitch_54" : 55,
"Note_Pitch_55" : 56,
"Note_Pitch_56" : 57,
"Note_Pitch_57" : 58,
"Note_Pitch_58" : 59,
"Note_Pitch_59" : 60,
"Note_Pitch_60" : 61,
"Note_Pitch_61" : 62,
"Note_Pitch_62" : 63,
"Note_Pitch_63" : 64,
"Note_Pitch_64" : 65,
"Note_Pitch_65" : 66,
"Note_Pitch_66" : 67,
"Note_Pitch_67" : 68,
"Note_Pitch_68" : 69,
"Note_Pitch_69" : 70,
"Note_Pitch_70" : 71,
"Note_Pitch_71" : 72,
"Note_Pitch_72" : 73,
"Note_Pitch_73" : 74,
"Note_Pitch_74" : 75,
"Note_Pitch_75" : 76,
"Note_Pitch_76" : 77,
"Note_Pitch_77" : 78,
"Note_Pitch_78" : 79,
"Note_Pitch_79" : 80,
"Note_Pitch_80" : 81,
"Note_Pitch_81" : 82,
"Note_Pitch_82" : 83,
"Note_Pitch_83" : 84,
"Note_Pitch_84" : 85,
"Note_Pitch_85" : 86,
"Note_Pitch_86" : 87,
"Note_Pitch_87" : 88,
"Note_Pitch_88" : 89,
"Note_Pitch_89" : 90,
"Note_Pitch_90" : 91,
"Note_Pitch_91" : 92,
"Note_Pitch_92" : 93,
"Note_Pitch_93" : 94,
"Note_Pitch_94" : 95,
"Note_Pitch_95" : 96,
"Note_Pitch_96" : 97,
"Note_Pitch_97" : 98,
"Note_Pitch_98" : 99,
"Note_Pitch_99" : 100,
"Note_Pitch_100": 101,
"Note_Pitch_101": 102,
"Note_Pitch_102": 103,
"Note_Pitch_103": 104,
"Note_Pitch_104": 105,
"Note_Pitch_105": 106,
"Note_Pitch_106": 107,
"Note_Pitch_107": 108,
"Note_Pitch_108": 109,
"Note_Pitch_109": 110,
"Note_Pitch_110": 111,
"Note_Pitch_111": 112,
"Note_Pitch_112": 113,
"Note_Pitch_113": 114,
"Note_Pitch_114": 115,
"Note_Pitch_115": 116,
"Note_Pitch_116": 117,
"Note_Pitch_117": 118,
"Note_Pitch_118": 119,
"Note_Pitch_119": 120,
"Note_Pitch_120": 121,
"Note_Pitch_121": 122,
"Note_Pitch_122": 123,
"Note_Pitch_123": 124,
"Note_Pitch_124": 125,
"Note_Pitch_125": 126,
"Note_Pitch_126": 127,
"Note_Pitch_127": 128,
},
"duration" : {
0 : 0,
"Note_Duration_0" : 1,
"Note_Duration_120" : 2,
"Note_Duration_240" : 3,
"Note_Duration_360" : 4,
"Note_Duration_480" : 5,
"Note_Duration_600" : 6,
"Note_Duration_720" : 7,
"Note_Duration_840" : 8,
"Note_Duration_960" : 9,
"Note_Duration_1080": 10,
"Note_Duration_1200": 11,
"Note_Duration_1320": 12,
"Note_Duration_1440": 13,
"Note_Duration_1560": 14,
"Note_Duration_1680": 15,
"Note_Duration_1800": 16,
"Note_Duration_1920": 17
},
"velocity" : {
0 : 0,
"Note_Velocity_0" : 1,
"Note_Velocity_1" : 2,
"Note_Velocity_2" : 3,
"Note_Velocity_3" : 4,
"Note_Velocity_4" : 5,
"Note_Velocity_5" : 6,
"Note_Velocity_6" : 7,
"Note_Velocity_7" : 8,
"Note_Velocity_8" : 9,
"Note_Velocity_9" : 10,
"Note_Velocity_10" : 11,
"Note_Velocity_11" : 12,
"Note_Velocity_12" : 13,
"Note_Velocity_13" : 14,
"Note_Velocity_14" : 15,
"Note_Velocity_15" : 16,
"Note_Velocity_16" : 17,
"Note_Velocity_17" : 18,
"Note_Velocity_18" : 19,
"Note_Velocity_19" : 20,
"Note_Velocity_20" : 21,
"Note_Velocity_21" : 22,
"Note_Velocity_22" : 23,
"Note_Velocity_23" : 24,
"Note_Velocity_24" : 25,
"Note_Velocity_25" : 26,
"Note_Velocity_26" : 27,
"Note_Velocity_27" : 28,
"Note_Velocity_28" : 29,
"Note_Velocity_29" : 30,
"Note_Velocity_30" : 31,
"Note_Velocity_31" : 32,
"Note_Velocity_32" : 33,
"Note_Velocity_33" : 34,
"Note_Velocity_34" : 35,
"Note_Velocity_35" : 36,
"Note_Velocity_36" : 37,
"Note_Velocity_37" : 38,
"Note_Velocity_38" : 39,
"Note_Velocity_39" : 40,
"Note_Velocity_40" : 41,
"Note_Velocity_41" : 42,
"Note_Velocity_42" : 43,
"Note_Velocity_43" : 44,
"Note_Velocity_44" : 45,
"Note_Velocity_45" : 46,
"Note_Velocity_46" : 47,
"Note_Velocity_47" : 48,
"Note_Velocity_48" : 49,
"Note_Velocity_49" : 50,
"Note_Velocity_50" : 51,
"Note_Velocity_51" : 52,
"Note_Velocity_52" : 53,
"Note_Velocity_53" : 54,
"Note_Velocity_54" : 55,
"Note_Velocity_55" : 56,
"Note_Velocity_56" : 57,
"Note_Velocity_57" : 58,
"Note_Velocity_58" : 59,
"Note_Velocity_59" : 60,
"Note_Velocity_60" : 61,
"Note_Velocity_61" : 62,
"Note_Velocity_62" : 63,
"Note_Velocity_63" : 64,
"Note_Velocity_64" : 65,
"Note_Velocity_65" : 66,
"Note_Velocity_66" : 67,
"Note_Velocity_67" : 68,
"Note_Velocity_68" : 69,
"Note_Velocity_69" : 70,
"Note_Velocity_70" : 71,
"Note_Velocity_71" : 72,
"Note_Velocity_72" : 73,
"Note_Velocity_73" : 74,
"Note_Velocity_74" : 75,
"Note_Velocity_75" : 76,
"Note_Velocity_76" : 77,
"Note_Velocity_77" : 78,
"Note_Velocity_78" : 79,
"Note_Velocity_79" : 80,
"Note_Velocity_80" : 81,
"Note_Velocity_81" : 82,
"Note_Velocity_82" : 83,
"Note_Velocity_83" : 84,
"Note_Velocity_84" : 85,
"Note_Velocity_85" : 86,
"Note_Velocity_86" : 87,
"Note_Velocity_87" : 88,
"Note_Velocity_88" : 89,
"Note_Velocity_89" : 90,
"Note_Velocity_90" : 91,
"Note_Velocity_91" : 92,
"Note_Velocity_92" : 93,
"Note_Velocity_93" : 94,
"Note_Velocity_94" : 95,
"Note_Velocity_95" : 96,
"Note_Velocity_96" : 97,
"Note_Velocity_97" : 98,
"Note_Velocity_98" : 99,
"Note_Velocity_99" : 100,
"Note_Velocity_100": 101,
"Note_Velocity_101": 102,
"Note_Velocity_102": 103,
"Note_Velocity_103": 104,
"Note_Velocity_104": 105,
"Note_Velocity_105": 106,
"Note_Velocity_106": 107,
"Note_Velocity_107": 108,
"Note_Velocity_108": 109,
"Note_Velocity_109": 110,
"Note_Velocity_110": 111,
"Note_Velocity_111": 112,
"Note_Velocity_112": 113,
"Note_Velocity_113": 114,
"Note_Velocity_114": 115,
"Note_Velocity_115": 116,
"Note_Velocity_116": 117,
"Note_Velocity_117": 118,
"Note_Velocity_118": 119,
"Note_Velocity_119": 120,
"Note_Velocity_120": 121,
"Note_Velocity_121": 122,
"Note_Velocity_122": 123,
"Note_Velocity_123": 124,
"Note_Velocity_124": 125,
"Note_Velocity_125": 126,
"Note_Velocity_126": 127,
"Note_Velocity_127": 128,
"Note_Velocity_128": 129,
"Note_Velocity_129": 130,
"Note_Velocity_130": 131,
"Note_Velocity_131": 132,
"Note_Velocity_132": 133,
"Note_Velocity_133": 134,
"Note_Velocity_134": 135,
"Note_Velocity_135": 136,
"Note_Velocity_136": 137,
"Note_Velocity_137": 138,
"Note_Velocity_138": 139,
"Note_Velocity_139": 140,
"Note_Velocity_140": 141,
"Note_Velocity_141": 142,
"Note_Velocity_142": 143,
"Note_Velocity_143": 144,
"Note_Velocity_144": 145,
"Note_Velocity_145": 146,
"Note_Velocity_146": 147,
"Note_Velocity_147": 148,
"Note_Velocity_148": 149,
"Note_Velocity_149": 150,
"Note_Velocity_150": 151,
"Note_Velocity_151": 152,
"Note_Velocity_152": 153,
"Note_Velocity_153": 154,
"Note_Velocity_154": 155,
"Note_Velocity_155": 156,
"Note_Velocity_156": 157,
"Note_Velocity_157": 158,
"Note_Velocity_158": 159,
"Note_Velocity_159": 160,
"Note_Velocity_160": 161,
"Note_Velocity_161": 162,
"Note_Velocity_162": 163,
"Note_Velocity_163": 164,
"Note_Velocity_164": 165,
"Note_Velocity_165": 166,
"Note_Velocity_166": 167,
"Note_Velocity_167": 168,
"Note_Velocity_168": 169,
"Note_Velocity_169": 170,
"Note_Velocity_170": 171,
"Note_Velocity_171": 172,
"Note_Velocity_172": 173,
"Note_Velocity_173": 174,
"Note_Velocity_174": 175,
"Note_Velocity_175": 176,
"Note_Velocity_176": 177,
"Note_Velocity_177": 178,
"Note_Velocity_178": 179,
"Note_Velocity_179": 180,
"Note_Velocity_180": 181,
"Note_Velocity_181": 182,
"Note_Velocity_182": 183,
"Note_Velocity_183": 184,
"Note_Velocity_184": 185,
"Note_Velocity_185": 186,
"Note_Velocity_186": 187,
"Note_Velocity_187": 188,
"Note_Velocity_188": 189,
"Note_Velocity_189": 190,
"Note_Velocity_190": 191,
"Note_Velocity_191": 192,
"Note_Velocity_192": 193,
"Note_Velocity_193": 194,
"Note_Velocity_194": 195,
"Note_Velocity_195": 196,
"Note_Velocity_196": 197,
"Note_Velocity_197": 198,
"Note_Velocity_198": 199,
"Note_Velocity_199": 200,
"Note_Velocity_200": 201,
"Note_Velocity_201": 202,
"Note_Velocity_202": 203,
"Note_Velocity_203": 204,
"Note_Velocity_204": 205,
"Note_Velocity_205": 206,
"Note_Velocity_206": 207,
"Note_Velocity_207": 208,
"Note_Velocity_208": 209,
"Note_Velocity_209": 210,
"Note_Velocity_210": 211,
"Note_Velocity_211": 212,
"Note_Velocity_212": 213,
"Note_Velocity_213": 214,
"Note_Velocity_214": 215,
"Note_Velocity_215": 216,
"Note_Velocity_216": 217,
"Note_Velocity_217": 218,
"Note_Velocity_218": 219,
"Note_Velocity_219": 220,
"Note_Velocity_220": 221,
"Note_Velocity_221": 222,
"Note_Velocity_222": 223,
"Note_Velocity_223": 224,
"Note_Velocity_224": 225,
"Note_Velocity_225": 226,
"Note_Velocity_226": 227,
"Note_Velocity_227": 228,
"Note_Velocity_228": 229,
"Note_Velocity_229": 230,
"Note_Velocity_230": 231,
"Note_Velocity_231": 232,
"Note_Velocity_232": 233,
"Note_Velocity_233": 234,
"Note_Velocity_234": 235,
"Note_Velocity_235": 236,
"Note_Velocity_236": 237,
"Note_Velocity_237": 238,
"Note_Velocity_238": 239,
"Note_Velocity_239": 240,
"Note_Velocity_240": 241,
"Note_Velocity_241": 242,
"Note_Velocity_242": 243,
"Note_Velocity_243": 244,
"Note_Velocity_244": 245,
"Note_Velocity_245": 246,
"Note_Velocity_246": 247,
"Note_Velocity_247": 248,
"Note_Velocity_248": 249,
"Note_Velocity_249": 250,
"Note_Velocity_250": 251,
"Note_Velocity_251": 252,
"Note_Velocity_252": 253,
"Note_Velocity_253": 254,
"Note_Velocity_254": 255,
"Note_Velocity_255": 256,
"Note_Velocity_256": 257,
"Note_Velocity_257": 258,
"Note_Velocity_258": 259,
"Note_Velocity_259": 260,
"Note_Velocity_260": 261,
"Note_Velocity_261": 262,
"Note_Velocity_262": 263,
"Note_Velocity_263": 264,
"Note_Velocity_264": 265,
"Note_Velocity_265": 266,
"Note_Velocity_266": 267,
"Note_Velocity_267": 268,
"Note_Velocity_268": 269,
"Note_Velocity_269": 270,
"Note_Velocity_270": 271,
"Note_Velocity_271": 272,
"Note_Velocity_272": 273,
"Note_Velocity_273": 274,
"Note_Velocity_274": 275,
"Note_Velocity_275": 276,
"Note_Velocity_276": 277,
"Note_Velocity_277": 278,
"Note_Velocity_278": 279,
"Note_Velocity_279": 280,
"Note_Velocity_280": 281,
"Note_Velocity_281": 282,
"Note_Velocity_282": 283,
"Note_Velocity_283": 284,
"Note_Velocity_284": 285,
"Note_Velocity_285": 286,
"Note_Velocity_286": 287,
"Note_Velocity_287": 288,
"Note_Velocity_288": 289,
"Note_Velocity_289": 290,
"Note_Velocity_290": 291,
"Note_Velocity_291": 292,
"Note_Velocity_292": 293,
"Note_Velocity_293": 294,
"Note_Velocity_294": 295,
"Note_Velocity_295": 296,
"Note_Velocity_296": 297,
"Note_Velocity_297": 298,
"Note_Velocity_298": 299,
"Note_Velocity_299": 300,
"Note_Velocity_300": 301,
"Note_Velocity_301": 302,
"Note_Velocity_302": 303,
"Note_Velocity_303": 304,
"Note_Velocity_304": 305,
"Note_Velocity_305": 306,
"Note_Velocity_306": 307,
"Note_Velocity_307": 308,
"Note_Velocity_308": 309,
"Note_Velocity_309": 310,
"Note_Velocity_310": 311,
"Note_Velocity_311": 312,
"Note_Velocity_312": 313,
"Note_Velocity_313": 314,
"Note_Velocity_314": 315,
"Note_Velocity_315": 316,
"Note_Velocity_316": 317,
"Note_Velocity_317": 318,
"Note_Velocity_318": 319,
"Note_Velocity_319": 320,
"Note_Velocity_320": 321,
"Note_Velocity_321": 322,
"Note_Velocity_322": 323,
"Note_Velocity_323": 324,
"Note_Velocity_324": 325,
"Note_Velocity_325": 326,
"Note_Velocity_326": 327,
"Note_Velocity_327": 328,
"Note_Velocity_328": 329,
"Note_Velocity_329": 330,
"Note_Velocity_330": 331,
"Note_Velocity_331": 332,
"Note_Velocity_332": 333,
"Note_Velocity_333": 334,
"Note_Velocity_334": 335,
"Note_Velocity_335": 336,
"Note_Velocity_336": 337,
"Note_Velocity_337": 338,
"Note_Velocity_338": 339,
"Note_Velocity_339": 340,
"Note_Velocity_340": 341,
"Note_Velocity_341": 342,
"Note_Velocity_342": 343,
"Note_Velocity_343": 344,
"Note_Velocity_344": 345,
"Note_Velocity_345": 346,
"Note_Velocity_346": 347,
"Note_Velocity_347": 348,
"Note_Velocity_348": 349,
"Note_Velocity_349": 350,
"Note_Velocity_350": 351,
"Note_Velocity_351": 352,
"Note_Velocity_352": 353,
"Note_Velocity_353": 354,
"Note_Velocity_354": 355,
"Note_Velocity_355": 356,
"Note_Velocity_356": 357,
"Note_Velocity_357": 358,
"Note_Velocity_358": 359,
"Note_Velocity_359": 360,
"Note_Velocity_360": 361
},
"boundary" : {
0 : 0,
"None" : 1,
"Boundary": 2,
},
# 'density': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
'variance' : [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
}
preset_word2event = {
"tempo" : {
0 : 0,
1 : "CONTI",
2 : "Tempo_0",
3 : "Tempo_1",
4 : "Tempo_2",
5 : "Tempo_3",
6 : "Tempo_4",
7 : "Tempo_5",
8 : "Tempo_6",
9 : "Tempo_7",
10 : "Tempo_8",
11 : "Tempo_9",
12 : "Tempo_10",
13 : "Tempo_11",
14 : "Tempo_12",
15 : "Tempo_13",
16 : "Tempo_14",
17 : "Tempo_15",
18 : "Tempo_16",
19 : "Tempo_17",
20 : "Tempo_18",
21 : "Tempo_19",
22 : "Tempo_20",
23 : "Tempo_21",
24 : "Tempo_22",
25 : "Tempo_23",
26 : "Tempo_24",
27 : "Tempo_25",
28 : "Tempo_26",
29 : "Tempo_27",
30 : "Tempo_28",
31 : "Tempo_29",
32 : "Tempo_30",
33 : "Tempo_31",
34 : "Tempo_32",
35 : "Tempo_33",
36 : "Tempo_34",
37 : "Tempo_35",
38 : "Tempo_36",
39 : "Tempo_37",
40 : "Tempo_38",
41 : "Tempo_39",
42 : "Tempo_40",
43 : "Tempo_41",
44 : "Tempo_42",
45 : "Tempo_43",
46 : "Tempo_44",
47 : "Tempo_45",
48 : "Tempo_46",
49 : "Tempo_47",
50 : "Tempo_48",
51 : "Tempo_49",
52 : "Tempo_50",
53 : "Tempo_51",
54 : "Tempo_52",
55 : "Tempo_53",
56 : "Tempo_54",
57 : "Tempo_55",
58 : "Tempo_56",
59 : "Tempo_57",
60 : "Tempo_58",
61 : "Tempo_59",
62 : "Tempo_60",
63 : "Tempo_61",
64 : "Tempo_62",
65 : "Tempo_63",
66 : "Tempo_64",
67 : "Tempo_65",
68 : "Tempo_66",
69 : "Tempo_67",
70 : "Tempo_68",
71 : "Tempo_69",
72 : "Tempo_70",
73 : "Tempo_71",
74 : "Tempo_72",
75 : "Tempo_73",
76 : "Tempo_74",
77 : "Tempo_75",
78 : "Tempo_76",
79 : "Tempo_77",
80 : "Tempo_78",
81 : "Tempo_79",
82 : "Tempo_80",
83 : "Tempo_81",
84 : "Tempo_82",
85 : "Tempo_83",
86 : "Tempo_84",
87 : "Tempo_85",
88 : "Tempo_86",
89 : "Tempo_87",
90 : "Tempo_88",
91 : "Tempo_89",
92 : "Tempo_90",
93 : "Tempo_91",
94 : "Tempo_92",
95 : "Tempo_93",
96 : "Tempo_94",
97 : "Tempo_95",
98 : "Tempo_96",
99 : "Tempo_97",
100: "Tempo_98",
101: "Tempo_99",
102: "Tempo_100",
103: "Tempo_101",
104: "Tempo_102",
105: "Tempo_103",
106: "Tempo_104",
107: "Tempo_105",
108: "Tempo_106",
109: "Tempo_107",
110: "Tempo_108",
111: "Tempo_109",
112: "Tempo_110",
113: "Tempo_111",
114: "Tempo_112",
115: "Tempo_113",
116: "Tempo_114",
117: "Tempo_115",
118: "Tempo_116",
119: "Tempo_117",
120: "Tempo_118",
121: "Tempo_119",
122: "Tempo_120",
123: "Tempo_121",
124: "Tempo_122",
125: "Tempo_123",
126: "Tempo_124",
127: "Tempo_125",
128: "Tempo_126",
129: "Tempo_127",
130: "Tempo_128",
131: "Tempo_129",
132: "Tempo_130",
133: "Tempo_131",
134: "Tempo_132",
135: "Tempo_133",
136: "Tempo_134",
137: "Tempo_135",
138: "Tempo_136",
139: "Tempo_137",
140: "Tempo_138",
141: "Tempo_139",
142: "Tempo_140",
143: "Tempo_141",
144: "Tempo_142",
145: "Tempo_143",
146: "Tempo_144",
147: "Tempo_145",
148: "Tempo_146",
149: "Tempo_147",
150: "Tempo_148",
151: "Tempo_149",
152: "Tempo_150",
153: "Tempo_151",
154: "Tempo_152",
155: "Tempo_153",
156: "Tempo_154",
157: "Tempo_155",
158: "Tempo_156",
159: "Tempo_157",
160: "Tempo_158",
161: "Tempo_159",
162: "Tempo_160",
163: "Tempo_161",
164: "Tempo_162",
165: "Tempo_163",
166: "Tempo_164",
167: "Tempo_165",
168: "Tempo_166",
169: "Tempo_167",
170: "Tempo_168",
171: "Tempo_169",
172: "Tempo_170",
173: "Tempo_171",
174: "Tempo_172",
175: "Tempo_173",
176: "Tempo_174",
177: "Tempo_175",
178: "Tempo_176",
179: "Tempo_177",
180: "Tempo_178",
181: "Tempo_179",
182: "Tempo_180",
183: "Tempo_181",
184: "Tempo_182",
185: "Tempo_183",
186: "Tempo_184",
187: "Tempo_185",
188: "Tempo_186",
189: "Tempo_187",
190: "Tempo_188",
191: "Tempo_189",
192: "Tempo_190",
193: "Tempo_191",
194: "Tempo_192",
195: "Tempo_193",
196: "Tempo_194",
197: "Tempo_195",
198: "Tempo_196",
199: "Tempo_197",
200: "Tempo_198",
201: "Tempo_199",
202: "Tempo_200",
203: "Tempo_201",
204: "Tempo_202",
205: "Tempo_203",
206: "Tempo_204",
207: "Tempo_205",
208: "Tempo_206",
209: "Tempo_207",
210: "Tempo_208",
211: "Tempo_209",
212: "Tempo_210",
213: "Tempo_211",
214: "Tempo_212",
215: "Tempo_213",
216: "Tempo_214",
217: "Tempo_215",
218: "Tempo_216",
219: "Tempo_217",
220: "Tempo_218",
221: "Tempo_219",
222: "Tempo_220",
223: "Tempo_221",
224: "Tempo_222",
225: "Tempo_223",
226: "Tempo_224",
227: "Tempo_225",
228: "Tempo_226",
229: "Tempo_227",
230: "Tempo_228",
231: "Tempo_229",
232: "Tempo_230",
233: "Tempo_231",
234: "Tempo_232",
235: "Tempo_233",
236: "Tempo_234",
237: "Tempo_235",
238: "Tempo_236",
239: "Tempo_237",
240: "Tempo_238",
241: "Tempo_239",
242: "Tempo_240",
243: "Tempo_241",
244: "Tempo_242",
245: "Tempo_243",
246: "Tempo_244",
247: "Tempo_245",
248: "Tempo_246",
249: "Tempo_247",
250: "Tempo_248",
251: "Tempo_249",
252: "Tempo_250",
253: "Tempo_251",
254: "Tempo_252",
255: "Tempo_253",
256: "Tempo_254",
257: "Tempo_255",
258: "Tempo_256",
259: "Tempo_257",
260: "Tempo_258",
261: "Tempo_259",
262: "Tempo_260",
263: "Tempo_261",
264: "Tempo_262",
265: "Tempo_263",
266: "Tempo_264",
267: "Tempo_265",
268: "Tempo_266",
269: "Tempo_267",
270: "Tempo_268",
271: "Tempo_269",
272: "Tempo_270",
273: "Tempo_271",
274: "Tempo_272",
275: "Tempo_273",
276: "Tempo_274",
277: "Tempo_275",
278: "Tempo_276",
279: "Tempo_277",
280: "Tempo_278",
281: "Tempo_279",
282: "Tempo_280",
283: "Tempo_281",
284: "Tempo_282",
285: "Tempo_283",
286: "Tempo_284",
287: "Tempo_285",
288: "Tempo_286",
289: "Tempo_287",
290: "Tempo_288",
291: "Tempo_289",
292: "Tempo_290",
293: "Tempo_291",
294: "Tempo_292",
295: "Tempo_293",
296: "Tempo_294",
297: "Tempo_295",
298: "Tempo_296",
299: "Tempo_297",
300: "Tempo_298",
301: "Tempo_299",
302: "Tempo_300",
303: "Tempo_301",
304: "Tempo_302",
305: "Tempo_303",
306: "Tempo_304",
307: "Tempo_305",
308: "Tempo_306",
309: "Tempo_307",
310: "Tempo_308",
311: "Tempo_309",
312: "Tempo_310",
313: "Tempo_311",
314: "Tempo_312",
315: "Tempo_313",
316: "Tempo_314",
317: "Tempo_315",
318: "Tempo_316",
319: "Tempo_317",
320: "Tempo_318",
321: "Tempo_319",
322: "Tempo_320",
323: "Tempo_321",
324: "Tempo_322",
325: "Tempo_323",
326: "Tempo_324",
327: "Tempo_325",
328: "Tempo_326",
329: "Tempo_327",
330: "Tempo_328",
331: "Tempo_329",
332: "Tempo_330",
333: "Tempo_331",
334: "Tempo_332",
335: "Tempo_333",
336: "Tempo_334",
337: "Tempo_335",
338: "Tempo_336",
339: "Tempo_337",
340: "Tempo_338",
341: "Tempo_339",
342: "Tempo_340",
343: "Tempo_341",
344: "Tempo_342",
345: "Tempo_343",
346: "Tempo_344",
347: "Tempo_345",
348: "Tempo_346",
349: "Tempo_347",
350: "Tempo_348",
351: "Tempo_349",
352: "Tempo_350",
353: "Tempo_351",
354: "Tempo_352",
355: "Tempo_353",
356: "Tempo_354",
357: "Tempo_355",
358: "Tempo_356",
359: "Tempo_357",
360: "Tempo_358",
361: "Tempo_359",
362: "Tempo_360"
},
"chord" : {
0: "0",
1: "CONTI"
},
"bar-beat" : {
0 : 0,
1 : "Bar",
2 : "Beat_0",
3 : "Beat_1",
4 : "Beat_2",
5 : "Beat_3",
6 : "Beat_4",
7 : "Beat_5",
8 : "Beat_6",
9 : "Beat_7",
10: "Beat_8",
11: "Beat_9",
12: "Beat_10",
13: "Beat_11",
14: "Beat_12",
15: "Beat_13",
16: "Beat_14",
17: "Beat_15"
},
"type" : {
0: "EOS",
1: "Metrical",
2: "Note",
3: "Seg",
},
"instr_type": {
0: 'None',
1: 'Drums',
2: 'Piano',
3: 'Guitar',
4: 'Bass',
5: 'Strings',
},
"pitch" : {
0 : 0,
1 : "Note_Pitch_0",
2 : "Note_Pitch_1",
3 : "Note_Pitch_2",
4 : "Note_Pitch_3",
5 : "Note_Pitch_4",
6 : "Note_Pitch_5",
7 : "Note_Pitch_6",
8 : "Note_Pitch_7",
9 : "Note_Pitch_8",
10 : "Note_Pitch_9",
11 : "Note_Pitch_10",
12 : "Note_Pitch_11",
13 : "Note_Pitch_12",
14 : "Note_Pitch_13",
15 : "Note_Pitch_14",
16 : "Note_Pitch_15",
17 : "Note_Pitch_16",
18 : "Note_Pitch_17",
19 : "Note_Pitch_18",
20 : "Note_Pitch_19",
21 : "Note_Pitch_20",
22 : "Note_Pitch_21",
23 : "Note_Pitch_22",
24 : "Note_Pitch_23",
25 : "Note_Pitch_24",
26 : "Note_Pitch_25",
27 : "Note_Pitch_26",
28 : "Note_Pitch_27",
29 : "Note_Pitch_28",
30 : "Note_Pitch_29",
31 : "Note_Pitch_30",
32 : "Note_Pitch_31",
33 : "Note_Pitch_32",
34 : "Note_Pitch_33",
35 : "Note_Pitch_34",
36 : "Note_Pitch_35",
37 : "Note_Pitch_36",
38 : "Note_Pitch_37",
39 : "Note_Pitch_38",
40 : "Note_Pitch_39",
41 : "Note_Pitch_40",
42 : "Note_Pitch_41",
43 : "Note_Pitch_42",
44 : "Note_Pitch_43",
45 : "Note_Pitch_44",
46 : "Note_Pitch_45",
47 : "Note_Pitch_46",
48 : "Note_Pitch_47",
49 : "Note_Pitch_48",
50 : "Note_Pitch_49",
51 : "Note_Pitch_50",
52 : "Note_Pitch_51",
53 : "Note_Pitch_52",
54 : "Note_Pitch_53",
55 : "Note_Pitch_54",
56 : "Note_Pitch_55",
57 : "Note_Pitch_56",
58 : "Note_Pitch_57",
59 : "Note_Pitch_58",
60 : "Note_Pitch_59",
61 : "Note_Pitch_60",
62 : "Note_Pitch_61",
63 : "Note_Pitch_62",
64 : "Note_Pitch_63",
65 : "Note_Pitch_64",
66 : "Note_Pitch_65",
67 : "Note_Pitch_66",
68 : "Note_Pitch_67",
69 : "Note_Pitch_68",
70 : "Note_Pitch_69",
71 : "Note_Pitch_70",
72 : "Note_Pitch_71",
73 : "Note_Pitch_72",
74 : "Note_Pitch_73",
75 : "Note_Pitch_74",
76 : "Note_Pitch_75",
77 : "Note_Pitch_76",
78 : "Note_Pitch_77",
79 : "Note_Pitch_78",
80 : "Note_Pitch_79",
81 : "Note_Pitch_80",
82 : "Note_Pitch_81",
83 : "Note_Pitch_82",
84 : "Note_Pitch_83",
85 : "Note_Pitch_84",
86 : "Note_Pitch_85",
87 : "Note_Pitch_86",
88 : "Note_Pitch_87",
89 : "Note_Pitch_88",
90 : "Note_Pitch_89",
91 : "Note_Pitch_90",
92 : "Note_Pitch_91",
93 : "Note_Pitch_92",
94 : "Note_Pitch_93",
95 : "Note_Pitch_94",
96 : "Note_Pitch_95",
97 : "Note_Pitch_96",
98 : "Note_Pitch_97",
99 : "Note_Pitch_98",
100: "Note_Pitch_99",
101: "Note_Pitch_100",
102: "Note_Pitch_101",
103: "Note_Pitch_102",
104: "Note_Pitch_103",
105: "Note_Pitch_104",
106: "Note_Pitch_105",
107: "Note_Pitch_106",
108: "Note_Pitch_107",
109: "Note_Pitch_108",
110: "Note_Pitch_109",
111: "Note_Pitch_110",
112: "Note_Pitch_111",
113: "Note_Pitch_112",
114: "Note_Pitch_113",
115: "Note_Pitch_114",
116: "Note_Pitch_115",
117: "Note_Pitch_116",
118: "Note_Pitch_117",
119: "Note_Pitch_118",
120: "Note_Pitch_119",
121: "Note_Pitch_120",
122: "Note_Pitch_121",
123: "Note_Pitch_122",
124: "Note_Pitch_123",
125: "Note_Pitch_124",
126: "Note_Pitch_125",
127: "Note_Pitch_126",
128: "Note_Pitch_127",
129: "Note_Pitch_128",
},
"duration" : {
0 : 0,
1 : "Note_Duration_0",
2 : "Note_Duration_120",
3 : "Note_Duration_240",
4 : "Note_Duration_360",
5 : "Note_Duration_480",
6 : "Note_Duration_600",
7 : "Note_Duration_720",
8 : "Note_Duration_840",
9 : "Note_Duration_960",
10: "Note_Duration_1080",
11: "Note_Duration_1200",
12: "Note_Duration_1320",
13: "Note_Duration_1440",
14: "Note_Duration_1560",
15: "Note_Duration_1680",
16: "Note_Duration_1800",
17: "Note_Duration_1920"
},
"velocity" : {
0 : 0,
1 : "Note_Velocity_0",
2 : "Note_Velocity_1",
3 : "Note_Velocity_2",
4 : "Note_Velocity_3",
5 : "Note_Velocity_4",
6 : "Note_Velocity_5",
7 : "Note_Velocity_6",
8 : "Note_Velocity_7",
9 : "Note_Velocity_8",
10 : "Note_Velocity_9",
11 : "Note_Velocity_10",
12 : "Note_Velocity_11",
13 : "Note_Velocity_12",
14 : "Note_Velocity_13",
15 : "Note_Velocity_14",
16 : "Note_Velocity_15",
17 : "Note_Velocity_16",
18 : "Note_Velocity_17",
19 : "Note_Velocity_18",
20 : "Note_Velocity_19",
21 : "Note_Velocity_20",
22 : "Note_Velocity_21",
23 : "Note_Velocity_22",
24 : "Note_Velocity_23",
25 : "Note_Velocity_24",
26 : "Note_Velocity_25",
27 : "Note_Velocity_26",
28 : "Note_Velocity_27",
29 : "Note_Velocity_28",
30 : "Note_Velocity_29",
31 : "Note_Velocity_30",
32 : "Note_Velocity_31",
33 : "Note_Velocity_32",
34 : "Note_Velocity_33",
35 : "Note_Velocity_34",
36 : "Note_Velocity_35",
37 : "Note_Velocity_36",
38 : "Note_Velocity_37",
39 : "Note_Velocity_38",
40 : "Note_Velocity_39",
41 : "Note_Velocity_40",
42 : "Note_Velocity_41",
43 : "Note_Velocity_42",
44 : "Note_Velocity_43",
45 : "Note_Velocity_44",
46 : "Note_Velocity_45",
47 : "Note_Velocity_46",
48 : "Note_Velocity_47",
49 : "Note_Velocity_48",
50 : "Note_Velocity_49",
51 : "Note_Velocity_50",
52 : "Note_Velocity_51",
53 : "Note_Velocity_52",
54 : "Note_Velocity_53",
55 : "Note_Velocity_54",
56 : "Note_Velocity_55",
57 : "Note_Velocity_56",
58 : "Note_Velocity_57",
59 : "Note_Velocity_58",
60 : "Note_Velocity_59",
61 : "Note_Velocity_60",
62 : "Note_Velocity_61",
63 : "Note_Velocity_62",
64 : "Note_Velocity_63",
65 : "Note_Velocity_64",
66 : "Note_Velocity_65",
67 : "Note_Velocity_66",
68 : "Note_Velocity_67",
69 : "Note_Velocity_68",
70 : "Note_Velocity_69",
71 : "Note_Velocity_70",
72 : "Note_Velocity_71",
73 : "Note_Velocity_72",
74 : "Note_Velocity_73",
75 : "Note_Velocity_74",
76 : "Note_Velocity_75",
77 : "Note_Velocity_76",
78 : "Note_Velocity_77",
79 : "Note_Velocity_78",
80 : "Note_Velocity_79",
81 : "Note_Velocity_80",
82 : "Note_Velocity_81",
83 : "Note_Velocity_82",
84 : "Note_Velocity_83",
85 : "Note_Velocity_84",
86 : "Note_Velocity_85",
87 : "Note_Velocity_86",
88 : "Note_Velocity_87",
89 : "Note_Velocity_88",
90 : "Note_Velocity_89",
91 : "Note_Velocity_90",
92 : "Note_Velocity_91",
93 : "Note_Velocity_92",
94 : "Note_Velocity_93",
95 : "Note_Velocity_94",
96 : "Note_Velocity_95",
97 : "Note_Velocity_96",
98 : "Note_Velocity_97",
99 : "Note_Velocity_98",
100: "Note_Velocity_99",
101: "Note_Velocity_100",
102: "Note_Velocity_101",
103: "Note_Velocity_102",
104: "Note_Velocity_103",
105: "Note_Velocity_104",
106: "Note_Velocity_105",
107: "Note_Velocity_106",
108: "Note_Velocity_107",
109: "Note_Velocity_108",
110: "Note_Velocity_109",
111: "Note_Velocity_110",
112: "Note_Velocity_111",
113: "Note_Velocity_112",
114: "Note_Velocity_113",
115: "Note_Velocity_114",
116: "Note_Velocity_115",
117: "Note_Velocity_116",
118: "Note_Velocity_117",
119: "Note_Velocity_118",
120: "Note_Velocity_119",
121: "Note_Velocity_120",
122: "Note_Velocity_121",
123: "Note_Velocity_122",
124: "Note_Velocity_123",
125: "Note_Velocity_124",
126: "Note_Velocity_125",
127: "Note_Velocity_126",
128: "Note_Velocity_127",
129: "Note_Velocity_128",
130: "Note_Velocity_129",
131: "Note_Velocity_130",
132: "Note_Velocity_131",
133: "Note_Velocity_132",
134: "Note_Velocity_133",
135: "Note_Velocity_134",
136: "Note_Velocity_135",
137: "Note_Velocity_136",
138: "Note_Velocity_137",
139: "Note_Velocity_138",
140: "Note_Velocity_139",
141: "Note_Velocity_140",
142: "Note_Velocity_141",
143: "Note_Velocity_142",
144: "Note_Velocity_143",
145: "Note_Velocity_144",
146: "Note_Velocity_145",
147: "Note_Velocity_146",
148: "Note_Velocity_147",
149: "Note_Velocity_148",
150: "Note_Velocity_149",
151: "Note_Velocity_150",
152: "Note_Velocity_151",
153: "Note_Velocity_152",
154: "Note_Velocity_153",
155: "Note_Velocity_154",
156: "Note_Velocity_155",
157: "Note_Velocity_156",
158: "Note_Velocity_157",
159: "Note_Velocity_158",
160: "Note_Velocity_159",
161: "Note_Velocity_160",
162: "Note_Velocity_161",
163: "Note_Velocity_162",
164: "Note_Velocity_163",
165: "Note_Velocity_164",
166: "Note_Velocity_165",
167: "Note_Velocity_166",
168: "Note_Velocity_167",
169: "Note_Velocity_168",
170: "Note_Velocity_169",
171: "Note_Velocity_170",
172: "Note_Velocity_171",
173: "Note_Velocity_172",
174: "Note_Velocity_173",
175: "Note_Velocity_174",
176: "Note_Velocity_175",
177: "Note_Velocity_176",
178: "Note_Velocity_177",
179: "Note_Velocity_178",
180: "Note_Velocity_179",
181: "Note_Velocity_180",
182: "Note_Velocity_181",
183: "Note_Velocity_182",
184: "Note_Velocity_183",
185: "Note_Velocity_184",
186: "Note_Velocity_185",
187: "Note_Velocity_186",
188: "Note_Velocity_187",
189: "Note_Velocity_188",
190: "Note_Velocity_189",
191: "Note_Velocity_190",
192: "Note_Velocity_191",
193: "Note_Velocity_192",
194: "Note_Velocity_193",
195: "Note_Velocity_194",
196: "Note_Velocity_195",
197: "Note_Velocity_196",
198: "Note_Velocity_197",
199: "Note_Velocity_198",
200: "Note_Velocity_199",
201: "Note_Velocity_200",
202: "Note_Velocity_201",
203: "Note_Velocity_202",
204: "Note_Velocity_203",
205: "Note_Velocity_204",
206: "Note_Velocity_205",
207: "Note_Velocity_206",
208: "Note_Velocity_207",
209: "Note_Velocity_208",
210: "Note_Velocity_209",
211: "Note_Velocity_210",
212: "Note_Velocity_211",
213: "Note_Velocity_212",
214: "Note_Velocity_213",
215: "Note_Velocity_214",
216: "Note_Velocity_215",
217: "Note_Velocity_216",
218: "Note_Velocity_217",
219: "Note_Velocity_218",
220: "Note_Velocity_219",
221: "Note_Velocity_220",
222: "Note_Velocity_221",
223: "Note_Velocity_222",
224: "Note_Velocity_223",
225: "Note_Velocity_224",
226: "Note_Velocity_225",
227: "Note_Velocity_226",
228: "Note_Velocity_227",
229: "Note_Velocity_228",
230: "Note_Velocity_229",
231: "Note_Velocity_230",
232: "Note_Velocity_231",
233: "Note_Velocity_232",
234: "Note_Velocity_233",
235: "Note_Velocity_234",
236: "Note_Velocity_235",
237: "Note_Velocity_236",
238: "Note_Velocity_237",
239: "Note_Velocity_238",
240: "Note_Velocity_239",
241: "Note_Velocity_240",
242: "Note_Velocity_241",
243: "Note_Velocity_242",
244: "Note_Velocity_243",
245: "Note_Velocity_244",
246: "Note_Velocity_245",
247: "Note_Velocity_246",
248: "Note_Velocity_247",
249: "Note_Velocity_248",
250: "Note_Velocity_249",
251: "Note_Velocity_250",
252: "Note_Velocity_251",
253: "Note_Velocity_252",
254: "Note_Velocity_253",
255: "Note_Velocity_254",
256: "Note_Velocity_255",
257: "Note_Velocity_256",
258: "Note_Velocity_257",
259: "Note_Velocity_258",
260: "Note_Velocity_259",
261: "Note_Velocity_260",
262: "Note_Velocity_261",
263: "Note_Velocity_262",
264: "Note_Velocity_263",
265: "Note_Velocity_264",
266: "Note_Velocity_265",
267: "Note_Velocity_266",
268: "Note_Velocity_267",
269: "Note_Velocity_268",
270: "Note_Velocity_269",
271: "Note_Velocity_270",
272: "Note_Velocity_271",
273: "Note_Velocity_272",
274: "Note_Velocity_273",
275: "Note_Velocity_274",
276: "Note_Velocity_275",
277: "Note_Velocity_276",
278: "Note_Velocity_277",
279: "Note_Velocity_278",
280: "Note_Velocity_279",
281: "Note_Velocity_280",
282: "Note_Velocity_281",
283: "Note_Velocity_282",
284: "Note_Velocity_283",
285: "Note_Velocity_284",
286: "Note_Velocity_285",
287: "Note_Velocity_286",
288: "Note_Velocity_287",
289: "Note_Velocity_288",
290: "Note_Velocity_289",
291: "Note_Velocity_290",
292: "Note_Velocity_291",
293: "Note_Velocity_292",
294: "Note_Velocity_293",
295: "Note_Velocity_294",
296: "Note_Velocity_295",
297: "Note_Velocity_296",
298: "Note_Velocity_297",
299: "Note_Velocity_298",
300: "Note_Velocity_299",
301: "Note_Velocity_300",
302: "Note_Velocity_301",
303: "Note_Velocity_302",
304: "Note_Velocity_303",
305: "Note_Velocity_304",
306: "Note_Velocity_305",
307: "Note_Velocity_306",
308: "Note_Velocity_307",
309: "Note_Velocity_308",
310: "Note_Velocity_309",
311: "Note_Velocity_310",
312: "Note_Velocity_311",
313: "Note_Velocity_312",
314: "Note_Velocity_313",
315: "Note_Velocity_314",
316: "Note_Velocity_315",
317: "Note_Velocity_316",
318: "Note_Velocity_317",
319: "Note_Velocity_318",
320: "Note_Velocity_319",
321: "Note_Velocity_320",
322: "Note_Velocity_321",
323: "Note_Velocity_322",
324: "Note_Velocity_323",
325: "Note_Velocity_324",
326: "Note_Velocity_325",
327: "Note_Velocity_326",
328: "Note_Velocity_327",
329: "Note_Velocity_328",
330: "Note_Velocity_329",
331: "Note_Velocity_330",
332: "Note_Velocity_331",
333: "Note_Velocity_332",
334: "Note_Velocity_333",
335: "Note_Velocity_334",
336: "Note_Velocity_335",
337: "Note_Velocity_336",
338: "Note_Velocity_337",
339: "Note_Velocity_338",
340: "Note_Velocity_339",
341: "Note_Velocity_340",
342: "Note_Velocity_341",
343: "Note_Velocity_342",
344: "Note_Velocity_343",
345: "Note_Velocity_344",
346: "Note_Velocity_345",
347: "Note_Velocity_346",
348: "Note_Velocity_347",
349: "Note_Velocity_348",
350: "Note_Velocity_349",
351: "Note_Velocity_350",
352: "Note_Velocity_351",
353: "Note_Velocity_352",
354: "Note_Velocity_353",
355: "Note_Velocity_354",
356: "Note_Velocity_355",
357: "Note_Velocity_356",
358: "Note_Velocity_357",
359: "Note_Velocity_358",
360: "Note_Velocity_359",
361: "Note_Velocity_360"
},
"boundary" : {
0: 0,
1: "None",
2: "Boundary",
},
"key" : {
0 : 0,
1 : "None",
2 : 'C',
3 : 'C#',
4 : 'D',
5 : 'D#',
6 : 'E',
7 : 'F',
8 : 'F#',
9 : 'G',
10: 'G#',
11: 'A',
12: 'A#',
13: 'B',
14: 'c',
15: 'c#',
16: 'd',
17: 'd#',
18: 'e',
19: 'f',
20: 'f#',
21: 'g',
22: 'g#',
23: 'a',
24: 'a#',
25: 'b',
}
}
================================================
FILE: src/video2npz/metadata2numpy_mix.py
================================================
#/usr/bin/env python
# -*- coding: UTF-8 -*-
import json
import os
import math
import argparse
import numpy as np
from dictionary_mix import preset_event2word
from stat_mix import vbeat_weight_percentile, fmpb_percentile
RESOLUTION = 16
DIMENSION = {
'beat' : 0,
'density' : 1,
'strength': 2,
'i_beat' : 3,
'n_beat' : 4,
'p_beat' : 5,
}
N_DIMENSION = len(DIMENSION)
def _cal_density(flow_magnitude):
for i, percentile in enumerate(fmpb_percentile):
if flow_magnitude < percentile:
return i
return len(fmpb_percentile)
def _cal_strength(weight):
for i, percentile in enumerate(vbeat_weight_percentile):
if weight < percentile:
return i
return len(vbeat_weight_percentile)
def _get_beat_token(beat, strength, i_beat, n_beat):
l = [[0] * N_DIMENSION]
l[0][DIMENSION['beat']] = preset_event2word['beat']['Beat_%d' % beat]
l[0][DIMENSION['strength']] = strength
l[0][DIMENSION['i_beat']] = i_beat
l[0][DIMENSION['n_beat']] = n_beat
l[0][DIMENSION['p_beat']] = round(float(i_beat) / n_beat * 100) + 1
return l
def _get_bar_token(density, i_beat, n_beat):
l = [[0] * N_DIMENSION]
l[0][DIMENSION['beat']] = preset_event2word['beat']['Bar']
l[0][DIMENSION['density']] = density + 1
l[0][DIMENSION['i_beat']] = i_beat
l[0][DIMENSION['n_beat']] = n_beat
l[0][DIMENSION['p_beat']] = round(float(i_beat) / n_beat * 100) + 1
return l
def metadata2numpy(metadata):
vbeats = metadata['vbeats']
fmpb = metadata['flow_magnitude_per_bar']
n_beat = int(math.ceil(float(metadata['duration']) / 60 * float(metadata['tempo']) * 4))
n_bars = 0 # 已添加 bar token 个数
l = []
for vbeat in vbeats:
# add bar token
while int(vbeat['bar']) >= n_bars:
i_beat = n_bars * RESOLUTION
l += _get_bar_token(density=_cal_density(fmpb[n_bars]), i_beat=i_beat, n_beat=n_beat)
n_bars += 1
# add beat token
i_beat = int(vbeat['bar']) * RESOLUTION + int(vbeat['tick'])
l += _get_beat_token(beat=int(vbeat['tick']), strength=_cal_strength(vbeat['weight']), i_beat=i_beat,
n_beat=n_beat)
# add empty bars
while n_bars < len(fmpb):
i_beat = n_bars * RESOLUTION
l += _get_bar_token(density=_cal_density(fmpb[n_bars]), i_beat=i_beat, n_beat=n_beat)
n_bars += 1
return np.asarray(l, dtype=int)
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--out_dir', default="../../inference/")
parser.add_argument('--video', default="final_640.mp4")
parser.add_argument('--metadata', default="metadata.json")
args = parser.parse_args()
video_name = os.path.basename(args.video)
with open(args.metadata) as f:
metadata = json.load(f)
target_path = os.path.join(args.out_dir, video_name.replace('.mp4', '.npz'))
print('processing to save to %s' % target_path)
input_numpy = metadata2numpy(metadata)
np.savez(target_path, input=input_numpy)
print("saved to " + str(target_path))
================================================
FILE: src/video2npz/optical_flow.py
================================================
#encoding=utf-8
import cv2
import argparse
import os
import numpy as np
import matplotlib.pyplot as plt
import skvideo.io
from tqdm import tqdm
def makedirs(dirs):
for dir in dirs:
if not os.path.exists(dir):
os.makedirs(dir)
video_dir = '../../videos/'
flow_dir = 'flow/'
fig_dir = 'fig/'
makedirs([video_dir, flow_dir, fig_dir, 'optical_flow/'])
TIME_PER_BAR = 2 # 暂定2s一小节
# 文字参数
ORG = (50, 50)
FONT = cv2.FONT_HERSHEY_SIMPLEX
FONT_SCALE = 1
COLOR = (0, 0, 255)
THICKNESS = 2
def dense_optical_flow(method, video_path, params=[], to_gray=False):
# print(video_path)
assert os.path.exists(video_path)
metadata = skvideo.io.ffprobe(video_path)
print("video loaded successfully")
frame, time = metadata['video']['@avg_frame_rate'].split('/')
fps = round(float(frame) / float(time))
if os.path.exists(os.path.join(flow_dir, os.path.basename(video_path).split('.')[0] + '.npz')):
flow_magnitude_list = list(
np.load(os.path.join(flow_dir, os.path.basename(video_path).split('.')[0] + '.npz'))['flow'])
else:
# Read the video and first frame
video = skvideo.io.vread(video_path)[:]
n_frames = len(video) # 总帧数
old_frame = video[0]
# crate HSV & make Value a constant
hsv = np.zeros_like(old_frame)
hsv[..., 1] = 255
# Preprocessing for exact method
if to_gray:
old_frame = cv2.cvtColor(old_frame, cv2.COLOR_RGB2GRAY)
flow_magnitude_list = []
for i in tqdm(range(1, n_frames)):
# Read the next frame
new_frame = video[i]
# Preprocessing for exact method
if to_gray:
new_frame = cv2.cvtColor(new_frame, cv2.COLOR_RGB2GRAY)
# Calculate Optical Flow
flow = method(old_frame, new_frame, None, *params)
flow_magnitude = np.mean(np.abs(flow))
flow_magnitude_list.append(flow_magnitude)
# Update the previous frame
old_frame = new_frame
frame_per_bar = TIME_PER_BAR * fps
flow_magnitude_per_bar = []
temp = np.zeros((len(flow_magnitude_list)))
for i in np.arange(0, len(flow_magnitude_list), frame_per_bar):
mean_flow = np.mean(flow_magnitude_list[int(i): min(int(i + frame_per_bar), len(flow_magnitude_list))])
flow_magnitude_per_bar.append(mean_flow)
temp[int(i): min(int(i + frame_per_bar), len(flow_magnitude_list))] = mean_flow
np.savez(os.path.join(flow_dir, os.path.basename(video_path).split('.')[0] + '.npz'),
flow=np.asarray(flow_magnitude_list))
# 绘制flow强度折线图
x = np.arange(0, len(flow_magnitude_list))
plt.figure(figsize=(10, 4))
plt.plot(x, temp, 'b.')
plt.title('Optical Flow Magnitude')
plt.savefig(os.path.join(fig_dir, os.path.basename(video_path).split('.')[0] + '.jpg'))
# return optical_flow, flow_magnitude_list
return flow_magnitude_per_bar, flow_magnitude_list
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument("--method", choices=["farneback", "lucaskanade_dense", "rlof"], default="farneback")
parser.add_argument("--video", default="../../videos/final_640.mp4")
args = parser.parse_args()
flow = []
video_path = args.video
print("video_path", video_path)
if args.method == 'lucaskanade_dense':
method = cv2.optflow.calcOpticalFlowSparseToDense
optical_flow, flow_magnitude_list = dense_optical_flow(method, video_path, to_gray=True)
elif args.method == 'farneback':
method = cv2.calcOpticalFlowFarneback
params = [0.5, 3, 15, 3, 5, 1.2, 0] # default Farneback's algorithm parameters
optical_flow, flow_magnitude_list = dense_optical_flow(method, video_path, params, to_gray=True)
elif args.method == "rlof":
method = cv2.optflow.calcOpticalFlowDenseRLOF
optical_flow, flow_magnitude_list = dense_optical_flow(method, video_path)
flow += optical_flow
flow = np.asarray(flow)
for percentile in range(10, 101, 10):
print('percentile %d: %.4f' % (percentile, np.percentile(flow, percentile)))
np.savez('optical_flow/flow.npz', flow=flow)
================================================
FILE: src/video2npz/resize_video.py
================================================
import os
def resize_video(video_name, in_dir='video', out_dir='video_360p', max_height=360):
command = 'ffmpeg -i %s -strict -2 -vf scale=-1:%d %s -v quiet' % (
os.path.join(in_dir, video_name),
max_height,
os.path.join(out_dir, video_name)
)
print(command)
os.system(command)
return os.path.join(out_dir, video_name)
================================================
FILE: src/video2npz/resize_videos.sh
================================================
mkdir video_360p
files=$(ls video)
for filename in $files
do
echo $filename
ffmpeg -i video/$filename -strict -2 -vf scale=-1:360 video_360p/$filename -v quiet
done
================================================
FILE: src/video2npz/stat_mix.py
================================================
def _cal_density(flow_magnitude):
for i, percentile in enumerate(fmpb_percentile):
if flow_magnitude < percentile:
return i
return len(fmpb_percentile)
def _cal_strength(weight):
for i, percentile in enumerate(vbeat_weight_percentile):
if weight < percentile:
return i
return len(vbeat_weight_percentile)
# calculated based on a set of videos
vbeat_weight_percentile = [0, 0.22890276357193542, 0.4838207191278801, 0.7870981363596372, 0.891160136856027,
0.9645568135300789, 0.991241869205911, 0.9978208223154553, 0.9996656159745393,
0.9998905521344276]
fmpb_percentile = [0.008169269189238548, 0.020344337448477745, 0.02979462407529354, 0.041041795164346695,
0.07087484002113342, 0.10512548685073853, 0.14267262816429138, 0.19095642864704132,
0.5155120491981506, 0.7514784336090088, 0.9989343285560608, 1.2067525386810303, 1.6322582960128784,
2.031705141067505, 2.467430591583252, 2.8104422092437744]
================================================
FILE: src/video2npz/video2metadata.py
================================================
#/usr/bin/env python
# -*- coding: UTF-8 -*-
import matplotlib
matplotlib.use('Agg')
import visbeat3 as vb
import os
import os.path as osp
import cv2
import json
import argparse
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.pyplot import MultipleLocator
def makedirs(d):
if not osp.exists(d):
os.makedirs(d)
def frange(start, stop, step=1.0):
while start < stop:
yield start
start += step
def process_all_videos(args):
out_json = {}
for i, video_name in enumerate(os.listdir(args.video_dir)):
if '.mp4' not in video_name:
continue
print('%d/%d: %s' % (i, len(os.listdir(args.video_dir)), video_name))
metadata = process_video(video_name, args)
out_json[video_name] = metadata
json_str = json.dumps(out_json, indent=4)
with open(osp.join(args.video_dir, 'metadata.json'), 'w') as f:
f.write(json_str)
def process_video(video_path, args):
figsize = (32, 4)
dpi = 200
xrange = (0, 95)
x_major_locator = MultipleLocator(2)
vb.Video.getVisualTempo = vb.Video_CV.getVisualTempo
video = os.path.basename(video_path)
vlog = vb.PullVideo(name=video, source_location=osp.join(video_path), max_height=360)
vbeats = vlog.getVisualBeatSequences(search_window=None)[0]
tempo, beats = vlog.getVisualTempo()
print("Tempo is", tempo)
vbeats_list = []
for vbeat in vbeats:
i_beat = round(vbeat.start / 60 * tempo * 4)
vbeat_dict = {
'start_time': vbeat.start,
'bar' : int(i_beat // 16),
'tick' : int(i_beat % 16),
'weight' : vbeat.weight
}
if vbeat_dict['tick'] % args.resolution == 0: # only select vbeat that lands on the xth tick
vbeats_list.append(vbeat_dict)
print('%d / %d vbeats selected' % (len(vbeats_list), len(vbeats)))
npz = np.load("flow/" + video.replace('.mp4', '.npz'), allow_pickle=True)
print(npz.keys())
flow_magnitude_list = npz['flow']
fps = round(vlog.n_frames() / float(vlog.getDuration()))
fpb = int(round(fps * 4 * 60 / tempo)) # frame per bar
fmpb = [] # flow magnitude per bar
temp = np.zeros((len(flow_magnitude_list)))
for i in range(0, len(flow_magnitude_list), fpb):
mean_flow = np.mean(flow_magnitude_list[i: min(i + fpb, len(flow_magnitude_list))])
fmpb.append(float(mean_flow))
temp[i: min(i + fpb, len(flow_magnitude_list))] = mean_flow
if args.visualize:
makedirs('image')
height = vlog.getFrame(0).shape[0]
thumbnails = [vlog.getFrameFromTime(t)[:, :int(height * 2.5 / 10), :] for t in list(frange(25, 35, 1))]
thumbnails = np.concatenate(thumbnails, axis=1)
cv2.cvtColor(thumbnails, cv2.COLOR_RGB2BGR)
cv2.imwrite(osp.join('image', video + '_thumbnails_1' + '.png'), thumbnails)
plt.rcParams.update({'font.size': 14})
plt.figure(figsize=figsize, dpi=dpi)
plt.subplots_adjust(bottom=0.15)
x2_time = [float(item) / fps for item in list(range(len(flow_magnitude_list)))]
plt.plot(x2_time[::3], flow_magnitude_list[::3], '-', color='#fff056', alpha=0.75, label="Per Frame")
for i, fm in enumerate(fmpb):
x_frame = [i * fpb, (i + 1) * fpb - 1]
x_time = [x / fps for x in x_frame]
y_fm = [fm, fm]
if i == 0:
plt.plot(x_time, y_fm, 'r-', label='Per Bar', lw=3)
else:
plt.plot(x_time, y_fm, 'r-', lw=3)
if xrange is not None:
plt.xlim(xrange)
ax = plt.gca()
ax.xaxis.set_major_locator(x_major_locator)
plt.xlabel('Time (s)')
plt.ylabel('Optical Flow Magnitude')
plt.legend(loc="upper left")
plt.savefig(osp.join('image', video + '_flow' + '.eps'), format='eps', transparent=True)
plt.savefig(osp.join('image', video + '_flow' + '.png'), format='png', transparent=True)
vlog.printVisualBeatSequences(figsize=figsize, save_path=osp.join('image', video + '_visbeat' + '.eps'),
xrange=xrange, x_major_locator=x_major_locator)
return {
'duration' : vlog.getDuration(),
'tempo' : tempo,
'vbeats' : vbeats_list,
'flow_magnitude_per_bar': fmpb,
}
if __name__ == '__main__':
# vb.SetAssetsDir('.' + os.sep + 'VisBeatAssets' + os.sep)
parser = argparse.ArgumentParser()
parser.add_argument('--video', type=str, default='../../videos/final_640.mp4')
parser.add_argument('--visualize', action='store_true', default=True)
parser.add_argument('--resolution', type=int, default=1)
args = parser.parse_args()
metadata = process_video(args.video, args)
with open("metadata.json", "w") as f:
json.dump(metadata, f)
print("saved to metadata.json")
================================================
FILE: src/video2npz/video2npz.sh
================================================
# extract flow magnitude into optical_flow/flow.npz
python optical_flow.py --video $1
# convert video into metadata.json with flow magnitude
python video2metadata.py --video $1
# convert metadata into .npz under `inference/`
python metadata2numpy_mix.py --video $1
================================================
FILE: src/video2npz/visbeat3/LICENSE
================================================
SOFTWARE LICENSE AGREEMENT For Stanford Docket S18-164, "Visual Rhythm and Beat:
Automatic Synchronization of visual beats to motion or dance"
1. By downloading the software in this directory, you ("RECIPIENT") are
explicitly agreeing to this license agreement with THE BOARD OF TRUSTEES OF THE
LELAND STANFORD JUNIOR UNIVERSITY ("STANFORD"). Stanford has an assignment to
"Visual Rhythm and Beat: Automatic Synchronization of visual beats to motion or
dance" ("Software") which was developed in the laboratory of Professor Maneesh
Agrawala.
2. By accepting, receiving, and using Software, including any accompanying
information, materials or manuals, you are agreeing to be bound by the terms of
this Agreement. If you do not agree to the terms of this Agreement, do not
download the Software from this directory, and destroy any copies you may have
already downloaded.
3. STANFORD grants a royalty-free, nonexclusive, and nontransferable license to
use the Software furnished hereunder, upon the terms and conditions set out
below.
4. RECIPIENT acknowledges that the Software is a research tool still in the
development stage and that it is being supplied as is, without any accompanying
services, support or improvements from STANFORD. STANFORD makes no
representations and extends no warranties of any kind, either express or implied
other than set out in this agreement.
5. RECIPIENT agrees to use the Software solely for internal, non-commercial
purposes and shall not distribute or transfer it to another location or to any
other person without prior written permission from STANFORD.
6. RECIPIENT acknowledges that any programs created based on the Software will
be considered a derivative of Software and cannot be used commercially without a
commercial license from Stanford.
7. RECIPIENT may make modifications to the Software and integrate Software into
RECIPIENT's own software.
8. RECIPIENT may not further distribute Software without express written
permission of STANFORD. If permission to transfer the Software is given,
RECIPIENT warrants that RECIPIENT will not remove or export any part of the
Software from the United States except in full compliance with all United States
export regulations and other applicable laws.
9. RECIPIENT will use the Software in compliance with all applicable laws,
policies and regulations including, but not limited to, any approvals, informed
consent and patient confidentiality principles.
10. RECIPIENT will indemnify, hold harmless, and defend STANFORD against any
claim of any kind arising out of or related to the exercise of any rights
granted under this Agreement or the breach of this Agreement by RECIPIENT.
11. Title and copyright to the Software and any derivatives and any associated
documentation shall at all times remain with STANFORD, and RECIPIENT agrees to
preserve same.
12. If RECIPIENT plans to publish any peer reviewed papers, abstracts, or
similar publications, RECIPIENT agrees to acknowledge Software and its creators
in a manner consistent with academic (industry) practice.
13. RECIPIENT agrees to maintain the visbeat logo that appears on all outputs
from the software.
14. If RECIPIENT wishes to terminate this license, RECIPIENT shall destroy all
copies of Software.
================================================
FILE: src/video2npz/visbeat3/MANIFEST.in
================================================
# Include the README
include *.md
# Include the license file
include LICENSE
include LICENSE.pdf
# Include the data files
recursive-include visbeat3/assets *
graft visbeat3/assets
global-include *
================================================
FILE: src/video2npz/visbeat3/README.md
================================================
# visbeat3
This is a migration of [visbeat](http://abedavis.com/visualbeat/) from Python2 to Python3. All credits belong to the original author.
Note: This repo is under development, for any issue, please PR directly!
## Install
```
pip3 install visbeat3
```
or
```
pip3 install -e git+https://github.com/haofanwang/visbeat3.git#egg=visbeat3
```
## Usage
```
import visbeat3 as vb
```
## Reference
```
@inproceedings{davis2018visual,
title={Visual rhythm and beat},
author={Davis, Abe and Agrawala, Maneesh},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops},
pages={2532--2535},
year={2018}
}
```
================================================
FILE: src/video2npz/visbeat3/VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Data/Backups/VideoSource.json
================================================
{
"a_info": {
"AObjectType": "VideoSource",
"base_path": null,
"directory_path": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238",
"file_base_name": "VideoSource",
"file_ext": ".json",
"file_name": "VideoSource.json",
"file_path": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/VideoSource.json",
"source_type": "file"
},
"directories": {
"backup": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Data/Backups/",
"data": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Data/",
"features": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Data/Features/",
"temp": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Data/TEMP/",
"versions": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Versions/",
"warps": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Versions/Warps/"
},
"source_location": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Versions//Source/Full/wzk_vlog_beat_enhance1_track1238.mp4",
"versions_info": {
"Original": {
"360": {
"duration": 38.07140473807141,
"end_time": 38.07140473807141,
"meta_data": {
"audio_codec": "aac",
"codec": "h264",
"duration": 38.11,
"ffmpeg_version": "4.2.2 built with Apple clang version 11.0.0 (clang-1100.0.33.8)",
"fps": 29.97,
"nframes": Infinity,
"pix_fmt": "yuv420p",
"plugin": "ffmpeg",
"rotate": 0,
"size": [
204,
360
],
"source_size": [
204,
360
]
},
"name": "wzk_vlog_beat_enhance1_track1238_360",
"num_frames_total": 1141,
"path": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Versions/Original/maxheight_360/wzk_vlog_beat_enhance1_track1238.mp4",
"sampling_rate": 29.97,
"start_time": 0
},
"Full": {
"duration": 34.83483483483484,
"end_time": 34.83483483483484,
"meta_data": {
"audio_codec": "aac",
"codec": "h264",
"duration": 38.08,
"ffmpeg_version": "4.2.2 built with Apple clang version 11.0.0 (clang-1100.0.33.8)",
"fps": 29.97,
"nframes": Infinity,
"pix_fmt": "yuv420p(tv",
"plugin": "ffmpeg",
"rotate": 0,
"size": [
544,
960
],
"source_size": [
544,
960
]
},
"name": "wzk_vlog_beat_enhance1_track1238_Full",
"num_frames_total": 1044,
"path": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Versions/Source/Full/wzk_vlog_beat_enhance1_track1238.mp4",
"sampling_rate": 29.97,
"start_time": 0
}
}
},
"video_file_name": "wzk_vlog_beat_enhance1_track1238.mp4"
}
================================================
FILE: src/video2npz/visbeat3/VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/VideoSource.json
================================================
{
"a_info": {
"AObjectType": "VideoSource",
"base_path": null,
"directory_path": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238",
"file_base_name": "VideoSource",
"file_ext": ".json",
"file_name": "VideoSource.json",
"file_path": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/VideoSource.json",
"source_type": "file"
},
"directories": {
"backup": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Data/Backups/",
"data": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Data/",
"features": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Data/Features/",
"temp": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Data/TEMP/",
"versions": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Versions/",
"warps": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Versions/Warps/"
},
"source_location": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Versions//Source/Full/wzk_vlog_beat_enhance1_track1238.mp4",
"versions_info": {
"Original": {
"360": {
"duration": 38.07140473807141,
"end_time": 38.07140473807141,
"meta_data": {
"audio_codec": "aac",
"codec": "h264",
"duration": 38.11,
"ffmpeg_version": "4.2.2 built with Apple clang version 11.0.0 (clang-1100.0.33.8)",
"fps": 29.97,
"nframes": Infinity,
"pix_fmt": "yuv420p",
"plugin": "ffmpeg",
"rotate": 0,
"size": [
204,
360
],
"source_size": [
204,
360
]
},
"name": "wzk_vlog_beat_enhance1_track1238_360",
"num_frames_total": 1141,
"path": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Versions/Original/maxheight_360/wzk_vlog_beat_enhance1_track1238.mp4",
"sampling_rate": 29.97,
"start_time": 0
},
"Full": {
"duration": 34.83483483483484,
"end_time": 34.83483483483484,
"meta_data": {
"audio_codec": "aac",
"codec": "h264",
"duration": 38.08,
"ffmpeg_version": "4.2.2 built with Apple clang version 11.0.0 (clang-1100.0.33.8)",
"fps": 29.97,
"nframes": Infinity,
"pix_fmt": "yuv420p(tv",
"plugin": "ffmpeg",
"rotate": 0,
"size": [
544,
960
],
"source_size": [
544,
960
]
},
"name": "wzk_vlog_beat_enhance1_track1238_Full",
"num_frames_total": 1044,
"path": "./VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Versions/Source/Full/wzk_vlog_beat_enhance1_track1238.mp4",
"sampling_rate": 29.97,
"start_time": 0
}
}
},
"video_file_name": "wzk_vlog_beat_enhance1_track1238.mp4"
}
================================================
FILE: src/video2npz/visbeat3/bin/dancefer
================================================
#!/usr/bin/env python
import sys
import matplotlib
matplotlib.use('PS')
import visbeat3
source_url = 'https://www.youtube.com/watch?v={}'.format(sys.argv[0]);
target_url = 'https://www.youtube.com/watch?v={}'.format(sys.argv[1]);
output_path = sys.argv[2];
result = visbeat3.AutoDancefer(source=source_url, target = target_url,
output_path = output_path,
synch_video_beat = 0,
synch_audio_beat = 0,
beat_offset = 64,
nbeats = 128);
result.play();
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/ADefines.py
================================================
# AD_DEBUG = 1;
AD_DEBUG = 0;
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/AFileManager.py
================================================
from .AObject import *
#import shutil
from distutils.dir_util import copy_tree
class AFileManager(AObject):
"""AFileManager (class): Manages assets. This should really be replaced with a database of some sort...
Attributes:
todo
"""
@staticmethod
def AOBJECT_TYPE():
return 'AFileManager';
def getJSONPath(self):
return self.getPath();
def __init__(self, path=None, clear_temp=None):
"""If you provide a directory, it will look for a existing AFileManager.json in that directory, or create one if it does not already exist.
If you provide a json, it will use that json, unless the json doesn't exist, in which case it will complain...
"""
AObject.__init__(self, path=path);
# self.initializeBlank();
self.initWithPath(path=path, clear_temp=clear_temp);
def initializeBlank(self):
AObject.initializeBlank(self);
self.directories = {};
def getJSONName(self):
return self.AOBJECT_TYPE()+".json";
def initWithPath(self, path=None, clear_temp=None):
oldpath = None
newpath = path;
if(path):
if(os.path.isfile(path)):
self.loadFromJSON(self.getJSONPath()); #assume path property is already set to 'path'
oldpath = self.getPath(); #whatever was in the json, having overwritten path property
elif(os.path.isdir(path)):
json_file_path = path+os.sep+self.getJSONName();
self.setPath(json_file_path);
if(os.path.isfile(self.getJSONPath())):
self.loadFromJSON(json_file_path);
oldpath = self.getPath();
newpath = json_file_path;
# self.setPath(file_path=json_file_path);
else:
newpath=self.getJSONPath()
self.writeToJSON(json_path=newpath);#no json file found, so we create one
else:
assert False, "Given AFileManager path is neither an existing directory or file! path: {} (AFileManager.py)".format(path)
self.setPath(file_path=newpath);
if(oldpath):
oldir = get_dir_from_path(pathstring(oldpath));
newdir = get_dir_from_path(pathstring(newpath));
if(oldir != newdir):
AWARN("FILEMANAGER FOUND FILE MOVED FROM:\n{}\nTO:\n{}\nUPDATING DIRECTORIES...".format(oldir,
newdir));
for d in self.directories:
dpth = self.directories[d];
if(dpth.startswith(oldir)):
dpthst = dpth.lstrip(oldir);
self.directories[d]=os.path.join(newdir,dpthst);
AWARN("{} updated to {}".format(dpth, self.directories[d]));
self.setDir('data', pathstring(self.getDirectoryPath()+os.sep+"Data"+os.sep));
self.setDir('backup', pathstring(self.getDir('data')+"Backups"+os.sep));
self.setDir('temp', pathstring(self.getDir('data')+"TEMP"+os.sep));
temp_dir = self.getDir('temp');
if(os.path.isdir(temp_dir) and (clear_temp)):
for the_file in os.listdir(temp_dir):
file_path = os.path.join(temp_dir, the_file);
try:
if os.path.isfile(file_path):
os.remove(file_path);
#os.unlink(file_path);
#elif os.path.isdir(file_path): shutil.rmtree(file_path)
except Exception as e:
print(e)
make_sure_path_exists(temp_dir);
#Video.VIDEO_TEMP_DIR = temp_dir;
def setDir(self, name, path):
# AWARN("setting {} to {}".format(name, path))
# assert(name is not 'log')
self.directories[name]=path;
make_sure_path_exists(path);
return path;
def addDir(self, name):
assert(name not in self.directories), "tried to add {} dir to AFileManager, but this dir is already set"
return self.setDir(name, pathstring(self.getDirectoryPath()+os.sep+name+os.sep));
def getDir(self, name):
# printDictionary(self.directories)
return self.directories.get(name);
def emptyDir(self, name):
dpth = self.getDir(name);
if(dpth is not None and os.path.isdir(dpth)):
shutil.rmtree(dpth);
make_sure_path_exists(dpth);
def deleteDir(self, name):
dpth = self.getDir(name);
if (dpth is not None and os.path.isdir(dpth)):
shutil.rmtree(dpth);
d = dict(self.directories);
del d[name];
self.directories=d;
def toDictionary(self):
d = AObject.toDictionary(self);
d['directories']=self.directories;
#serialize class specific members
return d;
def copyPathToDir(self, path_to_copy, dest_dir):
dest_path = self.getDir(dest_dir);
if(dest_path):
if(os.path.isdir(path_to_copy)):
copy_tree(src=path_to_copy, dst=dest_path);
elif(os.path.isfile(path_to_copy)):
shutil.copy2(path_to_copy, dest_path)
return;
def copyDirToPath(self, dir_to_copy, dest_path):
src_path = self.getDir(dir_to_copy);
if(src_path):
if(os.path.isdir(dest_path)):
copy_tree(src=src_path, dst=dest_path);
return;
@staticmethod
def copyRandomFractionOfFilesInSourceDir(source_dir, dest_dir, fraction=1.0, ext=None):
"""
Copies a random fraction of files in source directory... Wrote this for splitting training/test data in ML applications.
:param source_dir:
:param dest_dir:
:param fraction:
:param ext:
:return:
"""
directories = []
subdirnames = []
filepaths = [];
for filename in os.listdir(source_dir):
path = os.path.join(source_dir, filename)
if os.path.isdir(path):
directories.append(path)
subdirnames.append(filename)
else:
# namepart, extpart = os.path.splitext(filename);
if((ext is None) or filename.lower().endswith(ext)):
filepaths.append(path);
n_to_copy = int(len(filepaths)*fraction);
random_seed = 0;
random.seed(random_seed);
random.shuffle(filepaths);
copy_sources = filepaths[:n_to_copy];
for src, dst in zip(copy_sources, [dest_dir]*len(copy_sources)):
#print("src: {}\ndst: {}".format(src, dst));
shutil.copy2(src, dst);
for d in range(len(directories)):
subdest = pathstring(os.path.join(dest_dir,subdirnames[d])+os.sep);
make_sure_dir_exists(subdest);
AFileManager.copyRandomFractionOfFilesInSourceDir(source_dir=directories[d], dest_dir=subdest, fraction=fraction, ext=ext);
def initFromDictionary(self, d):
AObject.initFromDictionary(self, d);
self.directories = d['directories'];
def save(self):
if(os.path.isfile(self.getJSONPath())):
os.rename(self.getJSONPath(), self.getDir('backup')+os.sep+self.AOBJECT_TYPE()+".json");
self.writeToJSON(self.getJSONPath());
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/AFuncDict.py
================================================
from .AParamDict import *
import pickle as pickle
import os
class AFuncDict(AParamDict):
"""AFuncDict (class): Extends AParamDict so that functions can be assigned to features and called whenever
computing those features is necessary.
Attributes:
data: name -> value, params
feature funcs: name -> function for evaluating
"""
def __init__(self, owner=None, name=None, path=None):
AParamDict.__init__(self, owner=owner, name=name, path=path);
self.functions = {};
def getEntry(self, name=None, params=None, force_recompute=False):
d = self.data.get(name);
if((d is not None) and (not force_recompute)):
return d;
else:
f = self.getFunction(name=name);
if(f is not None):
if(params is not None):
if(not params.get('force_recompute')):
params.update(dict(force_recompute=force_recompute));
self.setValue(name=name, value=f(self=self.owner, **params), params=params, modified=True);
else:
self.setValue(name=name, value=f(self=self.owner, force_recompute=force_recompute), params=params, modified=True);
return self.data.get(name);
return None;
def getValue(self, name=None, params=None, force_recompute=False):
d = self.getEntry(name=name, params=params, force_recompute=force_recompute);
if(d is not None):
return d.get('value');
else:
return None;
def getParams(self, name=None):
d = self.data.get(name);
if(d is not None):
return d.get('params');
else:
return None;
def getFunction(self, name=None):
return self.functions.get(name);
def setValue(self, name, value=None, params=None, modified=True):
self.data[name]['value']=value;
self.data[name]['params']=params;
self.setEntryModified(name=name, is_modified=modified)
#self.data[name]['modified']=modified;
def setFunction(self, name, function=None):
self.functions[name]=function;
def saveEntry(self, name, path, force=False):
"""Save one entry to one file."""
if(self.data.get(name) is None):
return None;
if(self.isEntryModified(name=name) or force or (not os.path.isfile(path))):
#pickleToPath
f = open(path, 'wb');
pickle.dump(self.getEntry(name=name), f, protocol=2);
f.close();
self.setEntryModified(name=name, is_modified=False);
#assert(False), "should not be saving in this test";
return True;
def setEntryModified(self, name, is_modified=True):
self.data[name]['modified']=is_modified;
if(is_modified):
self.setModified(is_modified=True);
def isEntryModified(self, name):
entry = self.data.get(name);
if(entry is not None):
m=entry.get('modified');
if(m is not None):
return m;
else:
return True;
else:
assert(False), "checking mod bit on entry that does not exist"
def isModified(self):
return self.modified;
def setModified(self, is_modified):
self.modified=is_modified;
def save(self, path, force=False):
"""save all entries to one file."""
if(force or self.isModified()):
f = open(path, 'wb');
pickle.dump(self.data, f, protocol=2);
f.close();
self.setModified(is_modified=False);
#assert(False), "should not be saving in this test";
return True;
def loadEntry(self, name, path):
"""load one entry from one file."""
f=open(path, 'rb');
self.setEntry(name=name, d=pickle.load(f));
f.close();
self.setEntryModified(name=name, is_modified=False);
return True;
def load(self, path):
"""Load a set of entries all from one file."""
f=open(path, 'rb');
newd = pickle.load(f);
self.data.update(newd);
f.close();
return True;
def getKeyList(self):
return list(self.data.keys());
def getFunctionList(self):
return list(self.functions.keys());
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/AImports.py
================================================
#AImports
from . import ADefines as defines
import os
import os.path
import errno
import json
import pickle as pickle
import glob
import subprocess
from operator import truediv
import shutil
import time
from time import gmtime, strftime, localtime
import random
from . import fileui
try:
from termcolor import colored
def AWARN(message):
if (defines.AD_DEBUG):
print((colored(message, 'red')))
def AINFORM(message):
if(defines.AD_DEBUG):
print((colored(message, 'blue')))
except ImportError:
if (defines.AD_DEBUG):
print("You do not have termcolor installed (pip install termcolor). AWARN will just show as plain print statements when defines.AD_DEBUG==True...")
def AWARN(message):
if (defines.AD_DEBUG):
print(message);
def AINFORM(message):
if (defines.AD_DEBUG):
print(message);
def local_time_string():
return strftime("%Y-%m-%d_%H:%M:%S", localtime());
def get_temp_file_path(final_file_path="TEMP", temp_dir_path = None):
pparts = os.path.split(final_file_path);
destfolder = pparts[0]+os.sep;
tempdir = temp_dir_path;
if(tempdir is None):
tempdir='.';
destfolder=pathstring(tempdir+os.sep);
tempname = 'TEMP_'+pparts[1];
temptry = 0;
while(os.path.isfile(destfolder+tempname)):
temptry=temptry+1;
tempname = 'TEMP{}_'.format(temptry)+pparts[1];
return pathstring(destfolder+tempname);
def runningInNotebook():
try:
shell = get_ipython().__class__.__name__
if shell == 'ZMQInteractiveShell':
return True # Jupyter notebook or qtconsole
elif shell == 'TerminalInteractiveShell':
return False # Terminal running IPython
else:
return False # Other type (?)
except NameError:
return False # Probably standard Python interpreter
def getshellname():
try:
shell = get_ipython().__class__.__name__
return shell;
except NameError:
return False # Probably standard Python interpreter
def runningInSpyder():
return 'SpyderKernel' in str(get_ipython().kernel.__class__);
def pickleToPath(d, path):
# assert(False)
print("pickling")
f = open(path, 'wb');
pickle.dump(d, f, protocol=2);
f.close();
return True;
def unpickleFromPath(path):
f=open(path, 'rb');
d=pickle.load(f);
f.close();
return d;
def make_sure_path_exists(path):
try:
os.makedirs(path)
except OSError as exception:
if exception.errno != errno.EEXIST:
raise
def make_sure_dir_exists(path):
pparts = os.path.split(path);
destfolder = pparts[0]+os.sep;
try:
os.makedirs(destfolder)
except OSError as exception:
if exception.errno != errno.EEXIST:
raise
def safe_file_name(input_string):
return ''.join([i if ord(i) < 128 else '_' for i in input_string]);
def pathstring(path):
return path.replace(os.sep+os.sep, os.sep);
def is_interactive():
import __main__ as main
return not hasattr(main, '__file__')
def printOb(obj):
for attr in dir(obj):
print("#### Obj.%s = %s\n" % (attr, getattr(obj, attr)))
def pathstring(path):
return path.replace(os.sep+os.sep, os.sep);
def get_prepended_name_file_path(original_file_path, string_to_prepend):
pparts = os.path.split(original_file_path);
destfolder = pparts[0]+os.sep;
pname = string_to_prepend+pparts[1];
return pathstring(destfolder+pname);
def safe_file_name(input_string):
return ''.join([i if ord(i) < 128 else '_' for i in input_string]);
def change_extension(input_path, new_ext):
nameparts = os.path.splitext(input_path);
return nameparts[0]+new_ext;
def printDictionary(obj):
if type(obj) == dict:
for k, v in list(obj.items()):
if hasattr(v, '__iter__'):
print(k)
printDictionary(v)
else:
print('%s : %s' % (k, v))
elif type(obj) == list:
for v in obj:
if hasattr(v, '__iter__'):
printDictionary(v)
else:
print(v)
else:
print(obj)
def spotgt_shift_bit_length(x):
#smallest power of two greater than
return 1<<(x-1).bit_length()
def get_file_name_from_path(pth):
return os.path.split(pth)[1];
def get_dir_from_path(pth):
return (os.path.split(pth)[0]+os.sep);
def get_file_names_from_paths(pths):
r = [];
for p in pths:
r.append(get_file_name_from_path(p));
return r;
def writeDictionaryToJSON(d, json_path=None):
if(json_path):
with open(json_path, 'w') as outfile:
json.dump(d, outfile, sort_keys = True, indent = 4, ensure_ascii=False);
def vtt_to_srt(fileContents):
replacement = re.sub(r'([\d]+)\.([\d]+)', r'\1,\2', fileContents)
replacement = re.sub(r'WEBVTT\n\n', '', replacement)
replacement = re.sub(r'^\d+\n', '', replacement)
replacement = re.sub(r'\n\d+\n', '\n', replacement)
return replacement
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/AObject.py
================================================
#AO#labels#AObject
import os
import json
from .AParamDict import *
from . import fileui
class AObject(object):
"""AObject (class): This is a paarent class used to implement any comon serialization or typing we might want to do later
Attributes:
labels: dictionary of meta_data
save, load, and clear funcs: these are hooks for functions that manage the object's data on disk.
"""
AOBJECT_BASE_PATH=None;
def __init__(self, path=None, **kwargs):
self.initializeBlank();
if(path):
self.setPath(file_path=path, **kwargs);
def initializeBlank(self):
self.a_info = {'AObjectType': self.AOBJECT_TYPE()};
# self.a_data = AParamDict(owner=self, name='a_data');
self.save_func = None;
self.load_func = None;
self.clear_func = None;
def setPath(self, file_path=None, **kwargs):
if(file_path):
self.a_info['file_path'] = pathstring(file_path);
pparts = os.path.split(self.a_info['file_path']);
self.a_info['file_name'] = pparts[1];
self.a_info['directory_path']=pparts[0];
filename = self.a_info.get('file_name');
if(filename):
nameparts = os.path.splitext(filename);
self.a_info['file_base_name'] = nameparts[0];
self.a_info['file_ext'] = nameparts[1];
self.a_info['base_path'] = kwargs.get('base_path');
if(self.a_info['base_path'] is None):
self.a_info['base_path'] = AObject.AOBJECT_BASE_PATH;
def getPath(self):
if('file_path' in self.a_info):
return self.a_info['file_path'];
else:
return None;
def _showFile(self):
if(fileui.Show is not None):
fileui.Show(self.getPath());
def _open(self):
if(fileui.Open is not None):
fileui.Open(self.getPath());
def getRelativePath(self, base_path=None):
base = base_path;
if(base is None):
base = self.a_info.get('base_path');
if(base is None):
AWARN("Base path not set!");
return str(self.getPath());
def getFileName(self):
if('file_name' in self.a_info):
return self.a_info['file_name'];
else:
return None;
def getFileExtension(self):
if('file_ext' in self.a_info):
return self.a_info['file_ext'];
else:
return None;
def getDirectoryPath(self):
return self.a_info.get('directory_path')
def setInfo(self, label, value):
self.a_info[label]=value;
def getInfo(self, label):
return self.a_info.get(label);
def loadFromJSON(self, json_path=None):
if(json_path):
self.setPath(file_path=json_path);
if('file_path' in self.a_info):
json_text=open(self.a_info['file_path']).read();
d = json.loads(json_text);
self.initFromDictionary(d);
def writeToJSON(self, json_path=None):
#with open(jsonpath+self.name+'.json', 'w') as outfile:
if(not json_path):
json_path = self.a_info.get('file_path');
if(json_path):
with open(json_path, 'w') as outfile:
json.dump(self.toDictionary(), outfile, sort_keys = True, indent = 4, ensure_ascii=False);
def serializeInfo(self):
return self.a_info;
def save(self, features_to_save='all', overwrite=True, **kwargs):
if(self.save_func):
self.save_func(self, features_to_save=features_to_save, overwrite=overwrite, **kwargs);
else:
AWARN("SAVE FUNCTION HAS NOT BEEN PROVIDED FOR {} INSTANCE".format(self.AOBJECT_TYPE()));
def load(self, features_to_load=None, **kwargs):
if(self.load_func):
self.load_func(self, features_to_load=features_to_load, **kwargs);
else:
AWARN("LOAD FUNCTION HAS NOT BEEN PROVIDED FOR {} INSTANCE".format(self.AOBJECT_TYPE()));
########### "VIRTUAL" FUNCTIONS #############
@staticmethod
def AOBJECT_TYPE():
return 'AObject';
def toDictionary(self):
d = {'a_info': self.serializeInfo()};
return d;
def initFromDictionary(self, d):
self.a_info = d.get('a_info');
# ##Example of how these functions should be written in a subclass, here we call the subclass 'AssetManager'
# def AOBJECT_TYPE(self):
# return 'AssetManager';
#
# def toDictionary(self):
# d = AObject.toDictionary(self);
# #serialize class specific members
# return d;
#
# def initFromDictionary(self, d):
# AObject.initFromDictionary(self, d);
# #do class specific inits with d;
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/AParamDict.py
================================================
from .AImports import *
class AParamDict(object):
"""AParamDict (class): Dictionary that stores values and the parameters used to compute those values. I use this
class for two things. The first is to store parameters for reproducability. The second is to only recompute
values when functions are called with different parameters (some of this latter functionality is tied up in
code that isn't part of the Lite release).
Attributes:
data: name -> value, params
"""
def __init__(self, owner=None, name=None, path=None):
self.name=name;
self.data = {};
self.owner=owner;
self.modified=False;
def getEntry(self, name=None, params=None, force_recompute=False):
d = self.data.get(name);
if((d is not None) and (not force_recompute)):
return d;
return None;
def setEntry(self, name, d):
assert(name!='all' and name!='each'),"Entry named '{}' is reserved in AParamDict".format(name);
self.data[name]=d;
def removeEntry(self, name, assert_if_absent=True, set_modified = True):
if(assert_if_absent):
assert(name in self.data),"Tried to remove entry {} that was not already in {}".format(name, self.__class__);
popentry = self.data.pop(name, None);
if(set_modified):
self.setModified(set_modified);
return popentry;
def hasEntry(self, name=None):
return (self.data.get(name) is not None);
def getValue(self, name=None, params=None, force_recompute=False):
d = self.getEntry(name=name, params=params, force_recompute=force_recompute);
if(d is not None):
return d.get('value');
else:
return None;
def getParams(self, name=None):
d = self.data.get(name);
if(d is not None):
return d.get('params');
else:
return None;
def setValue(self, name, value=None, params=None, modified=True):
self.data[name]['value']=value;
self.data[name]['params']=params;
self.setEntryModified(name=name, is_modified=modified)
def saveEntry(self, name, path, force=False):
"""Save one entry to one file."""
if(self.hasEntry(name=name) is None):
return None;
if(self.isEntryModified(name=name) or force or (not os.path.isfile(path))):
f = open(path, 'wb');
pickle.dump(self.getEntry(name=name), f, protocol=2);
f.close();
self.setEntryModified(name=name, is_modified=False);
return True;
def setEntryModified(self, name, is_modified=True):
self.data[name]['modified']=is_modified;
if(is_modified):
self.setModified(is_modified=True);
def isEntryModified(self, name):
m=self.data[name].get('modified');
if(m is not None):
return m;
else:
return True;
def isModified(self):
return self.modified;
def setModified(self, is_modified):
self.modified=is_modified;
def save(self, path, force=False):
"""save all entries to one file."""
if(force or self.isModified()):
f = open(path, 'wb');
pickle.dump(self.data, f, protocol=2);
f.close();
self.setModified(is_modified=False);
return True;
def loadEntry(self, name, path):
"""load one entry from one file."""
f=open(path, 'rb');
self.setEntry(name=name, d=pickle.load(f));
f.close();
self.setEntryModified(name=name, is_modified=False);
return True;
def load(self, path):
"""Load a set of entries all from one file."""
f=open(path, 'rb');
newd = pickle.load(f);
self.data.update(newd);
f.close();
return True;
def getKeyList(self):
return list(self.data.keys());
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/Audio.py
================================================
from .EventList import *
from .TimeSignal1D import *
from scipy.io.wavfile import write
import librosa
import librosa.display
from scipy import signal, fftpack
class Audio(TimeSignal1D):
"""Audio (class): A sound, and a bunch of convenience functions to go with it.
Attributes:
x: the sound signal
sampling_rate: the sampling rate
"""
FEATURE_FUNCS = TimeSignal1D.FEATURE_FUNCS.copy();
def __init__(self, path=None, sampling_rate=None, x=None, name=None):
#VBObject.__init__(self, path=path);
TimeSignal1D.__init__(self, path=path, sampling_rate=sampling_rate, x=x);
#self.initializeBlank();
if(name is not None):
self.name = name;
if(path):
self.loadFile();
if(self.name is None):
self.name = self.getInfo('file_name')
# \w+)')``
re_code = None
# Compilede regec (:py:func:`re.compile`) to detect, if input URL is valid for current backend.
# Example: ``re.compile(r'^http://myvideo\.com/.*')``
re_detect = None
# Pattern in which the code is inserted.
# Example: ``http://myvideo.com?code=%s``
# :type: str
pattern_url = None
pattern_thumbnail_url = None
re_detect = re.compile(
r'^(http(s)?://)?(www\.|m\.)?youtu(\.?)be(\.com)?/.*', re.I
)
re_code = re.compile(
r'''youtu(\.?)be(\.com)?/ # match youtube's domains
(\#/)? # for mobile urls
(embed/)? # match the embed url syntax
(v/)?
(watch\?v=)? # match the youtube page url
(ytscreeningroom\?v=)?
(feeds/api/videos/)?
(user\S*[^\w\-\s])?
(?P[\w\-]{11})[a-z0-9;:@?&%=+/\$_.-]* # match and extract
''',
re.I | re.X
)
pattern_url = '{protocol}://www.youtube.com/embed/{code}'
pattern_thumbnail_url = '{protocol}://img.youtube.com/vi/{code}/{resolution}'
resolutions = [
'maxresdefault.jpg',
'sddefault.jpg',
'hqdefault.jpg',
'mqdefault.jpg',
]
def get_url(self):
"""
Returns URL folded from :py:data:`pattern_url` and parsed code.
"""
url = self.pattern_url.format(code=self.code, protocol=self.protocol)
url += '?' + self.query.urlencode() if self.query else ''
return url
def get_thumbnail_url(self):
"""
Returns thumbnail URL folded from :py:data:`pattern_thumbnail_url` and
parsed code.
:rtype: str
"""
return self.pattern_thumbnail_url.format(code=self.code,
protocol=self.protocol)
def _getCode(self):
match = self.re_code.search(self._source_location)
if match:
return match.group('code')
parsed_url = urllib.parse.urlparse(self._source_location)
parsed_qs = urllib.parse.parse_qs(parsed_url.query)
if 'v' in parsed_qs:
code = parsed_qs['v'][0]
elif 'video_id' in parsed_qs:
code = parsed_qs['video_id'][0]
else:
raise UnknownIdException('Cannot get ID from `{0}`'.format(self._source_location))
return code
def get_thumbnail_url(self):
"""
Returns thumbnail URL folded from :py:data:`pattern_thumbnail_url` and
parsed code.
:rtype: str
"""
for resolution in self.resolutions:
temp_thumbnail_url = self.pattern_thumbnail_url.format(
code=self.code, protocol=self.protocol, resolution=resolution)
if int(requests.head(temp_thumbnail_url).status_code) < 400:
return temp_thumbnail_url
return None
class FilePathURL(SourceURL):
@classmethod
def SourceLocationType(cls):
return 'file_path';
def __init__(self, source_location):
self.source_location_type = self.SourceLocationType()
self._source_location = source_location;
@classmethod
def is_valid(cls, source_location):
return os.path.isfile(source_location);
def _getCode(self):
name_parts = os.path.splitext(os.path.basename(self._source_location));
return name_parts[0];
SOURCE_LOCATION_TYPES = (
YoutubeURL,
FilePathURL,
)
def ParseSourseLocation(url):
"""
:param url:
:return:
"""
for backend in SOURCE_LOCATION_TYPES:
if backend.is_valid(url):
return backend(url)
raise UnknownBackendException
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/TimeSignal.py
================================================
from .VBObject import *
from .Event import *
import math
from operator import truediv
from .VisBeatImports import *
class TimeSignal(VBObject):
"""TimeSignal (class): A time signal, and a bunch of convenience functions to go with it.
Attributes:
sampling_rate: the sampling rate
visualizations dictionary of visualizations. (removed - too many dependencies)
"""
def VBBJECT_TYPE(self):
return 'TimeSignal';
def __init__(self, path=None, sampling_rate = None):
VBObject.__init__(self, path=path);
# self.initializeBlank();
# self.visualizations = AFuncDict(owner=self, name='visualizations');
# self.visualizations.functions.update(self.VIS_FUNCS);
if(sampling_rate):
self.sampling_rate=sampling_rate;
def initializeBlank(self):
VBObject.initializeBlank(self);
self.sampling_rate = None;
#
@property
def frame_rate(self):
return self._getFrameRate();
def _getFrameRate(self):
raise NotImplementedError;
#
def getSampleAtTime(self, f):
prev_sample = self.getSampleAtIndex(math.floor(f));
next_sample = self.getSampleAtIndex(math.ceil(f));
sample_progress = f-np.floor(f);
return (next_sample*sample_progress)+(prev_sample*(1.0-sample_progress));
def getSampleAtIndex(self, i):
return self.getTimeForIndex();
def getDuration(self):
assert(False), "getDuration must be implemented for subclass of TimeSignal"
def getSampleDuration(self):
return 1.0/self.sampling_rate;
def getTimeForIndex(self, i):
return i*self.getSampleDuration();
# def toDictionary(self):
# d = VBObject.toDictionary(self);
# assert(False), "haven't implemented toDictionary for {} yet".format(self.VBOBJECT_TYPE())
# #serialize class specific members
# return d;
# def initFromDictionary(self, d):
# VBObject.initFromDictionary(self, d);
# assert(False), "haven't implemented initFromDictionary for {} yet".format(self.VBOBJECT_TYPE())
# #do class specific inits with d;
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/TimeSignal1D.py
================================================
from .TimeSignal import *
import math
class TimeSignal1D(TimeSignal):
"""TimeSignal1D (class): A time signal, and a bunch of convenience functions to go with it.
Attributes:
sampling_rate: the sampling rate
x: the time signal
"""
def VBOJECT_TYPE(self):
return 'TimeSignal1D';
def __init__(self, path=None, sampling_rate = None, x=None):
TimeSignal.__init__(self, path=path, sampling_rate=sampling_rate);
if(x is not None):
self.x = x;
#self.initializeBlank(); # will be called by parent
def initializeBlank(self):
TimeSignal.initializeBlank(self);#YES KEEP call through weird loops
self.x = None;
def getSignal(self, resampled=False):
return self.x;
def getSignalSegment(self, time_range):
signal = self.getSignal();
seg_start = int(time_range[0]*self.sampling_rate);
seg_end = int(time_range[1]*self.sampling_rate);
return signal[seg_start:seg_end];
def getFullSignal(self):
return self.x;
def getSampleAtTime(self, f):
prev_sample = self.x[int(math.floor(f))];
next_sample = self.x[int(math.ceil(f))];
sample_progress = f-np.floor(f);
return (next_sample*sample_progress)+(prev_sample*(1.0-sample_progress));
def getSampleAtIndex(self, i):
return self.x[i];
def getDuration(self):
return truediv(len(self.getSignal()), self.sampling_rate);
def setValueRange(self, value_range=None):
if(value_range is None):
value_range = [0,1];
data = self.x[:];
currentscale = np.max(data)-np.min(data);
data = (data/currentscale)*(value_range[1]-value_range[0]);
data = data-np.min(data)+value_range[0]
self.x = data;
def setMaxAbsValue(self, max_abs_val=1.0):
data = self.x[:];
currentscale = np.max(np.fabs(data));
data = (data/currentscale)*max_abs_val;
self.x = data;
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/VBMIDI.py
================================================
# %load VisBeat/MidiParse.py
import mido
import numpy as np
import os
# from TimeSignal1D import *
from .Audio import *
from .Event import *
class VBMIDITrack(object):
def __init__(self, track, ticks_per_beat):
self.mido_track = track;
self.name = track.name;
# print('Track {}: {}'.format(i, track.name));
self.tempo = next((msg.tempo for msg in track if msg.type == 'set_tempo'), None);
self.ticks_per_beat = ticks_per_beat;
# self._get_note_on_times();
self.note_on_times = None;
def getBPM(self):
return mido.tempo2bpm(self.tempo);
def getNoteOnTimes(self, include_negative=None):
if(self.note_on_times is None or (include_negative)):
self._get_note_on_times(include_negative = include_negative);
return self.note_on_times;
def _get_note_on_times(self, include_negative = None):
"""
Gets the starting time of each note.
"""
note_times = []
current_time = 0;
for msg in self.mido_track:
if not msg.is_meta:
delta_time = mido.tick2second(msg.time, self.ticks_per_beat, self.tempo)
current_time += delta_time
if msg.type == 'note_on' and msg.velocity > 0:
if (include_negative or current_time>=0):
note_times.append(current_time);
self.note_on_times = np.array(note_times);
def get_note_durations(self, track_num):
note_durations = []
currently_played_notes = {}
current_time = 0
for msg in self.mido_track:
if not msg.is_meta:
delta_time = mido.tick2second(msg.time, self.ticks_per_beat, self.tempo)
current_time += delta_time
if msg.type == 'note_on' and msg.velocity > 0 and msg.note not in currently_played_notes:
currently_played_notes[msg.note] = current_time
#if len(currently_played_notes) > 1:
# print "Number of played notes have reached: ", len(currently_played_notes)
elif msg.type == 'note_off' or (msg.type == 'note_on' and msg.velocity == 0):
duration = current_time - currently_played_notes[msg.note]
note_durations.append((currently_played_notes[msg.note], duration))
currently_played_notes.pop(msg.note)
assert not bool(currently_played_notes), "Finished looping through all messages. There should not be any notes playing at this point."
#print "before sorting: ", note_durations
note_durations.sort(key=lambda x : x[0])
#print "after sorting: ", note_durations
return np.array([ x[1] for x in note_durations ])
def get_mouth_events(self):
# note_times = []
current_time = 0;
number_on = 0;
events = [];
open_buffer = 0.1;
close_buffer = 0.1
last_type = 0;
events.append(Event(start=0, type='mouth_close', weight=1));
for msg in self.mido_track:
if not msg.is_meta:
delta_time = mido.tick2second(msg.time, self.ticks_per_beat, self.tempo)
current_time += delta_time
last_time = events[-1].start;
passed = current_time-last_time;
if msg.type == 'note_on' and msg.velocity > 0:
if(last_type==0):
if(passed>open_buffer):
events.append(Event(start=current_time-open_buffer, type='mouth_closed', weight=0));
events.append(Event(start=current_time, type='mouth_open', weight = truediv(msg.velocity,127)));
number_on = number_on + 1;
last_type=1;
if(msg.type == 'note_off' or (msg.type=='note_on' and msg.velocity == 0)):
if (last_type == 1):
if (passed > close_buffer):
events.append(Event(start=current_time - close_buffer, type='mouth_opened', weight=0));
events.append(Event(start=current_time, type='mouth_close', weight = truediv(msg.velocity,127)));
number_on = number_on-1;
last_type=0;
# assert(number_on>-1);
# if(number_on==0):
return events;
# self.note_on_times = np.array(note_times);
def getNoteOnTimesAsAudio(self, sampling_rate = None, note_sound=None, n_seconds=None):
assert(n_seconds is not None), "must provide n seconds"
if(sampling_rate is None):
sampling_rate = 16000;
if(note_sound is None):
note_sound = Audio.getPing();
s = Audio.Silence(n_seconds=n_seconds, sampling_rate=sampling_rate, name=self.name)
s = s.getWithSoundAdded(note_sound, self.getNoteOnTimes());
return s;
class VBMIDI(TimeSignal1D):
def __init__(self, path=None):
TimeSignal1D.__init__(self, path=path);
self.midi_file = mido.MidiFile(path)
self.tracks = [];
for i, track in enumerate(self.midi_file.tracks):
self.tracks.append(VBMIDITrack(track, self.midi_file.ticks_per_beat));
def getNoteOnTimes(self, include_negative=None):
return self.tracks[-1].getNoteOnTimes(include_negative=include_negative);
def getMouthEvents(self):
return self.tracks[-1].get_mouth_events();
def getNoteOnTimesAsAudio(self, sampling_rate=None, note_sound=None):
s = self.tracks[-1].getNoteOnTimesAsAudio(sampling_rate = sampling_rate, note_sound = note_sound, n_seconds = self.midi_file.length);
if (s.name is None):
s.name = self.getInfo('file_name')
return s;
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/VBObject.py
================================================
import os
import json
from .VisBeatImports import *
from .AFuncDict import *
from .AObject import AObject
class VBObject(AObject):
"""VBObject (class): This is a paarent class used to implement common serialization and other functions. There ends
up being three different dictionaries of data.
a_info - for small labels and such. Part of AObject.
a_data - for data to be computed in experiments. I generally use this for things I don't want to automatically
save out to file.
features - these are features tied to the results of functions, and manager classes (e.g. VideoSource) will save
these to disk for future use.
FEATURE_FUNCS is a dictionary mapping the names of features to their corresponding functions.
"""
FEATURE_FUNCS={};
def __init__(self, path=None):
AObject.__init__(self, path=path);
def initializeBlank(self):
AObject.initializeBlank(self);
self.a_info.update({'VBObjectType': self.VBOBJECT_TYPE()});
self.features = AFuncDict(owner=self, name='features');
self.features.functions.update(self.FEATURE_FUNCS);
def saveFeature(self, name, path):
"""Subclasses can implement version of this that will check members for features if those features arent found here."""
return self.features.saveEntry(name=name, path=path);
def saveFeatures(self, path):
return self.features.save(path=path);
def loadFeature(self, name, path):
"""Subclasses can implement version of this that will check whether feature is registered before loading."""
return self.features.loadEntry(name=name, path=path);
def loadFeatures(self, path):
return self.features.load(path=path);
def getFeature(self,name, force_recompute=False, **kwargs):
"""Understood to get the value of a feature. can automatically recompute if feature has registered function."""
params = kwargs;
assert (not kwargs.get('params')), "STILL TRYING TO USE PARAMS INSTEAD OF KWARGS. FIX THIS";
return self.features.getValue(name=name, params=kwargs, force_recompute=force_recompute);
def getFeatureEntry(self, name, params=None, force_recompute=False):
return self.features.getEntry(name=name, params=params, force_recompute=force_recompute);
def getFeatureParams(self, name):
return self.features.getParams(name=name);
def setFeature(self, name, value, params=None):
rval = self.features.setEntry(name=name, d=dict(value=value, params=params));
self.features.setEntryModified(name=name, is_modified=True);
return rval;
def removeFeature(self, name, assert_if_absent=True, set_modified=True):
self.features.removeEntry(name=name, assert_if_absent=assert_if_absent, set_modified=set_modified);
def hasFeature(self, name):
"""Just checks to see if it's there."""
return self.features.hasEntry(name=name);
def getFeatureFunction(self, feature_name):
return self.features.getFunction(name=feature_name);
def getFeaturesList(self):
return self.features.getKeyList();
def getFeatureFunctionsList(self):
return self.features.getFunctionList();
def clearFeatureFiles(self, features_to_clear=None, **kwargs):
if(self.clear_feature_files_func):
self.clear_feature_files_func(self, features_to_clear=features_to_clear, **kwargs);
else:
VBWARN("CLEAR FEATURE FILES FUNCTION HAS NOT BEEN PROVIDED FOR {} INSTANCE".format(self.VBOBJECT_TYPE()));
########### VIRTUAL FUNCTIONS #############
def AOBJECT_TYPE(self):
return 'VBObject';
def VBOBJECT_TYPE(self):
return self.AOBJECT_TYPE();
# ##Example of how these functions should be written ina subclass, for 'AssetManager' class
# def VBOBJECT_TYPE(self):
# return 'AssetManager';
#
# def toDictionary(self):
# d = VBObject.toDictionary(self);
# #serialize class specific members
# return d;
#
# def initFromDictionary(self, d):
# VBObject.initFromDictionary(self, d);
# #do class specific inits with d;
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/Video.py
================================================
#VideoVersion#from VisBeatImports import *
from .AObject import *
from .TimeSignal import *
from .Audio import *
from .Warp import *
from .Image import *
import moviepy.editor as mpy
from moviepy.audio.AudioClip import AudioArrayClip as MPYAudioArrayClip
import sys
import math
from operator import truediv
def MPYWriteVideoFile(mpyclip, filename, **kwargs):
temp_audio_filename = get_temp_file_path(final_file_path='TEMP_'+filename+'.m4a', temp_dir_path=Video.VIDEO_TEMP_DIR);
return mpyclip.write_videofile(filename=filename, temp_audiofile= temp_audio_filename, audio_codec='aac', **kwargs);
class Video(TimeSignal):
"""Video (class): A video, and a bunch of convenience functions to go with it.
Attributes:
"""
VIDEO_TEMP_DIR = './'
FEATURE_FUNCS = TimeSignal.FEATURE_FUNCS.copy();
def AOBJECT_TYPE(self):
return 'Video';
def __init__(self, path=None, name=None, num_frames_total=None):
TimeSignal.__init__(self, path=path);
if(name):
self.name = name;
if(path):
self.loadFile(num_frames_total=num_frames_total);
def initializeBlank(self):
TimeSignal.initializeBlank(self);#YES KEEP
self.name = None;
self.sampling_rate = None;
self.audio=None;
self.reader = None;
self.writer = None;
self.num_frames_total=None;
self.reshape=None;
self.meta_data = None;
self.sampling_rate = None;
self.source = None;
# _gui is not saved
self._gui = None;
def _getFrameRate(self):
return self.sampling_rate;
# _gui is from TimeSignal
#
@property
def gui(self):
return self._getGui();
def _getGui(self):
return self._gui;
@gui.setter
def gui(self, value):
self._setGui(value);
def _setGui(self, value):
self._gui = value;
#
def getVersionInfo(self):
"""This is the info to be saved in asset version dictionary"""
d={};
d['name']=self.name;
d['sampling_rate']=self.sampling_rate;
d['num_frames_total']=self.num_frames_total;
d['duration']=self.getDuration();
d['start_time']=self.getStartTime();
d['end_time']=self.getEndTime();
d['meta_data']=self.meta_data;
return d;
def getName(self):
if(self.name is None):
return self.getInfo('file_name');
else:
return self.name;
def getTempDir(self):
if(self.source is not None):
return self.source.getDir('temp');
else:
return Video.VIDEO_TEMP_DIR;
def getStringForHTMLStreamingBase64(self):
svideo = io.open(self.getPath(), 'r+b').read()
encoded = base64.b64encode(svideo)
return "data:video/mp4;base64,{0}".format(encoded.decode('ascii'));
def n_frames(self):
if(not self.num_frames_total):
self.num_frames_total = self.calcNumFramesTotal();
return self.num_frames_total;
def getDuration(self):
return truediv(self.n_frames(), self.sampling_rate);
def getStartTime(self):
return 0;
def getEndTime(self):
return self.getDuration();
def getMPYClip(self, get_audio=True):
return mpy.VideoFileClip(self.getPath(), audio=get_audio);
def getAudio(self):
return self.audio;
def loadFile(self, file_path=None, num_frames_total=None):
if (file_path):
self.setPath(file_path=file_path);
if ('file_path' in self.a_info):
self.reader = imageio.get_reader(self.a_info['file_path'], 'ffmpeg');
self.meta_data = self.reader.get_meta_data();
self.sampling_rate = self.meta_data['fps'];
if (num_frames_total is not None):
self.num_frames_total = num_frames_total;
else:
self.num_frames_total = self.calcNumFramesTotal();
try:
self.audio = Audio(self.a_info['file_path']);
self.audio.name =self.name;
#except RuntimeError:
except Exception:
print(("Issue loading audio for {}".format(self.a_info['file_path'].encode('utf-8'))));
self.audio = Audio(sampling_rate=16000);
self.audio.x = np.zeros(int(np.ceil(self.audio.sampling_rate*self.getDuration())));
def openVideoWriter(self, output_file_path, fps=None):
if('outputs' not in self.a_info):
self.a_info['outputs'] = [];
out_fps = fps;
if(not out_fps):
out_fps=self.sampling_rate;
make_sure_dir_exists(output_file_path);
self.writer = imageio.get_writer(output_file_path, 'ffmpeg', macro_block_size = None, fps = out_fps);
self.a_info['outputs'].append(output_file_path);
def closeVideoWriter(self):
self.writer.close();
self.writer = None;
def getFrameShape(self):
fs=self.getInfo('frame_shape');
if(fs is not None):
return fs;
else:
self.setInfo(label='frame_shape', value=self.reader.get_data(0).shape);
return self.getInfo('frame_shape');
def calcNumFramesTotal(self):
#assert(False)
print(("Calculating frames for {}...".format(self.name)))
valid_frames = 0
example_frame = self.reader.get_data(0);
self.setInfo(label='frame_shape',value=example_frame.shape);
# https://stackoverflow.com/questions/54778001/how-to-to-tackle-overflowerror-cannot-convert-float-infinity-to-integer
#for i in range(1, self.reader.get_length()):
for i in range(1, self.reader.count_frames()):
try:
self.reader.get_data(i);
except imageio.core.format.CannotReadFrameError as e:
break
valid_frames += 1
print("Done.")
return valid_frames
def readFrameBasic(self, i):
"""You should basically never call this"""
fi=i;
if(fi<0):
fi=0;
if(fi>(self.num_frames_total-1)):
fi=(self.num_frames_total-1);
return np.asarray(self.reader.get_data(int(fi)));
def getFrame(self, f):
return self.readFrameBasic(round(f));
def getFrameFromTime(self, t):
f = t*self.sampling_rate;
return self.getFrame(f=f);
def getFrameLinearInterp(self, f):
if(isinstance(f,int) or f.is_integer()):
return self.readFrameBasic(int(f));
prev_frame = self.readFrameBasic(math.floor(f));
next_frame = self.readFrameBasic(math.ceil(f));
frame_progress = f-np.floor(f);
rframe = (next_frame*frame_progress)+(prev_frame*(1.0-frame_progress));
return rframe.astype(prev_frame.dtype);
def writeFrame(self, img):
if self.writer.closed:
print('ERROR: Vid writer object is closed.')
else:
self.writer.append_data(img.astype(np.uint8))
def play(self):
if(ISNOTEBOOK):
print("Playing video:")
video = io.open(self.getPath(), 'r+b').read()
encoded = base64.b64encode(video)
vidhtml=HTML(data=''''''.format(encoded.decode('ascii')));
IPython.display.display(vidhtml);
# return vidhtml;
else:
print("HOW TO PLAY VIDEO? NOT A NOTEBOOK.")
def show(self):
self.play();
def write(self, output_path, output_sampling_rate=None):
assert output_path, "MUST PROVIDE OUTPUT PATH FOR VIDEO"
sampling_rate = output_sampling_rate;
if(not sampling_rate):
sampling_rate=self.sampling_rate;
tempfilepath = get_temp_file_path(final_file_path=output_path, temp_dir_path=self.getTempDir());
self.openVideoWriter(output_file_path=tempfilepath, fps=output_sampling_rate);
duration = self.getDuration();
nsamples = sampling_rate*duration;
old_frame_time = truediv(1.0,self.sampling_rate);
frame_start_times = np.linspace(0,self.getDuration(),num=nsamples,endpoint=False);
frame_index_floats = frame_start_times*self.sampling_rate;
start_timer=time.time();
last_timer=start_timer;
fcounter=0;
for nf in range(len(frame_index_floats)):
self.writeFrame(self.getFrame(frame_index_floats[nf]));
fcounter+=1;
if(not (fcounter%50)):
if((time.time()-last_timer)>10):
last_timer=time.time();
print(("{}%% done after {} seconds...".format(100.0*truediv(fcounter,len(frame_index_floats)), last_timer-start_timer)));
self.closeVideoWriter();
rvid = Video.CreateFromVideoAndAudio(video_path=tempfilepath, audio_object=self.audio, output_path=output_path);
os.remove(tempfilepath);
return rvid;
def VideoClip(self, start=None, end=None, name=None):
from . import VideoClip
if(start is None):
start = 0;
if(end is None):
end = self.getDuration();
clip = VideoClip.VideoClip(video=self, start=start, end=end);
if(name):
clip.name=name;
return clip;
def getImageFromFrame(self, i):
rimage = Image(data=self.getFrame(i));
rimage.setInfo(label='parent_video', value=self.getInfo('file_path'));
rimage.setInfo(label='frame_number',value=i);
return rimage;
def getImageFromTime(self, t):
rimage = Image(data=self.getFrameFromTime(t));
rimage.setInfo(label='parent_video', value=self.getInfo('file_path'));
rimage.setInfo(label='frame_time', value=t);
return rimage;
# def getFrameShape(self):
# return self.getImageFromFrame(0).getShape();
def writeResolutionCopyFFMPEG(self, path, max_height=None):
if(max_height is None):
max_height=self.getFrameShape()[0];
mpc = self.getMPYClip();
clip_resized = mpc.resize(height=max_height); # make the height 360px ( According to moviePy documenation The width is then computed so that the width/height ratio is conserved.)
if((clip_resized.size[0]%2)>0):
clip_resized=clip_resized.crop(x1=0,width=clip_resized.size[0]-1);
# clip_resized.write_videofile(path);
MPYWriteVideoFile(clip_resized, path);
rvid = Video(path);
return rvid;
def writeFFMPEG(self, output_path):
mpc = self.getMPYClip();
# mpc.write_videofile(output_path, preset='fast', codec='mpeg4');
MPYWriteVideoFile(mpc, output_path, preset='fast', codec='mpeg4')
rvid = Video(output_path);
return rvid;
def writeResolutionCopy(self, path, max_height=None, reshape=None, input_sampling_rate_factor = None, output_sampling_rate=None):
print((self.getPath()))
if(input_sampling_rate_factor is not None):
original_sampling_rate = self.sampling_rate;
self.sampling_rate=input_sampling_rate_factor*self.sampling_rate;
original_audio_sampling_rate = self.audio.sampling_rate;
self.audio.sampling_rate=self.audio.sampling_rate*input_sampling_rate_factor;
output_path = path;
inshape = self.getFrameShape();
if(reshape==True):
imshape = [inshape[1],inshape[0],inshape[2]];
outshape = None;
if(max_height and (max_height10):
last_timer=time.time();
print(("{}%% done after {} seconds...".format(100.0*truediv(fcounter,len(frame_index_floats)), last_timer-start_timer)));
self.closeVideoWriter();
rvid = Video.CreateFromVideoAndAudio(video_path=tempfilepath, audio_object=self.audio, output_path=output_path);
os.remove(tempfilepath);
if(input_sampling_rate_factor is not None):
#return original_sampling_rate
self.sampling_rate=original_sampling_rate;
self.audio.sampling_rate = original_audio_sampling_rate;
return rvid;
def writeWarped(self, output_path, warp, output_sampling_rate=None, output_audio=None, bitrate=None, vbmark = True, max_time = None, **kwargs):
sampling_rate = output_sampling_rate;
if (not sampling_rate):
sampling_rate = self.sampling_rate;
duration = self.getDuration();
old_frame_time = truediv(1.0, self.sampling_rate);
target_start = warp.getTargetStart();
target_end = warp.getTargetEnd();
target_duration = target_end - target_start;
if(max_time is not None and target_duration>max_time):
target_duration = max_time;
target_end = target_start+max_time;
print((
"target start: {}\ntarget end: {}\ntarget duration: {}".format(target_start, target_end, target_duration)));
new_n_samples = target_duration * sampling_rate;
target_start_times = np.linspace(target_start, target_end, num=new_n_samples, endpoint=False);
unwarped_target_times = [];
for st in target_start_times:
unwarped_target_times.append(warp.warpTargetTime(st));
frame_index_floats = np.true_divide(np.array(unwarped_target_times), old_frame_time);
tempfilepath = get_temp_file_path(final_file_path=output_path, temp_dir_path=Video.VIDEO_TEMP_DIR);
self.openVideoWriter(output_file_path=tempfilepath, fps=output_sampling_rate, **kwargs);
start_timer = time.time();
last_timer = start_timer;
fcounter = 0;
if(vbmark):
vbmarker = self.getImageFromFrame(0)._vbmarker();
for nf in range(len(frame_index_floats)):
try:
nwfr = self.getFrame(frame_index_floats[nf]);
if(vbmark):
nwfrm = Image(data=nwfr);
nwfrm._splatAtPixCoord(**vbmarker);
nwfr = nwfrm._pixels_uint;
self.writeFrame(nwfr);
except ValueError:
print(("VALUE ERROR ON WRITEFRAME {}".format(frame_index_floats[nf])));
self.writeFrame(self.getFrame(math.floor(frame_index_floats[nf])));
fcounter += 1;
if (not (fcounter % 50)):
if ((time.time() - last_timer) > 10):
last_timer = time.time();
print(("{}%% done after {} seconds...".format(100.0 * truediv(fcounter, len(frame_index_floats)),
last_timer - start_timer)));
self.closeVideoWriter();
silent_warped_vid = Video(tempfilepath);
if (not output_audio):
output_audio = self.getAudio();
cropped_output_audio = output_audio.AudioClip(start=target_start, end=target_end);
if ((self.getInfo('max_height') is None) and (bitrate is None)):
use_bitrate = "20000k";
print(('Using bitrate of {}'.format(use_bitrate)));
rvid = Video.CreateFromVideoAndAudioObjects(video=silent_warped_vid, audio=cropped_output_audio,
output_path=output_path, bitrate=use_bitrate);
elif (bitrate is 'regular'):
rvid = Video.CreateFromVideoAndAudioObjects(video=silent_warped_vid, audio=cropped_output_audio,
output_path=output_path);
else:
rvid = Video.CreateFromVideoAndAudioObjects(video=silent_warped_vid, audio=cropped_output_audio,
output_path=output_path, bitrate=bitrate);
os.remove(tempfilepath);
rvid.setInfo(label='warp_used', value=warp);
return rvid;
def getVersionLabel(self):
return self.getInfo('version_label');
def getWarpsDir(self):
if(self.source is None):
return self.getTempDir();
version_label = self.getVersionLabel()
return self.source.getWarpsDir(version_label=version_label);
def getWithBeginningCroppedToAudio(self, target):
if(isinstance(target, Audio)):
B=target;
else:
B=target.getAudio();
A = self.getAudio();
AB = A.getShiftAmountTo(B);
BA = B.getShiftAmountTo(A);
if(AB n_audio_samples_sig):
nreps = math.ceil(truediv(n_audio_samples_in_vid, n_audio_samples_sig));
if (is_stereo):
audio_sig = np.concatenate(
(audio_sig, np.zeros((2, n_audio_samples_in_vid - n_audio_samples_sig))),
axis=1);
else:
audio_sig = np.tile(audio_sig, (int(nreps)));
audio_sig = audio_sig[:int(n_audio_samples_in_vid)];
if (is_stereo):
# reshapex = np.reshape(audio_sig, (audio_sig.shape[1], audio_sig.shape[0]), order='F');
reshapex = np.transpose(audio_sig);
audio_clip = MPYAudioArrayClip(reshapex, fps=audio_sampling_rate); # from a numeric arra
else:
reshapex = audio_sig.reshape(len(audio_sig), 1);
reshapex = np.concatenate((reshapex, reshapex), axis=1);
audio_clip = MPYAudioArrayClip(reshapex, fps=audio_sampling_rate); # from a numeric arra
video_clip = video_object.getMPYClip();
video_clip = video_clip.set_audio(audio_clip);
# video_clip.write_videofile(output_path,codec='libx264', write_logfile=False);
if (bitrate is None):
# video_clip.write_videofile(output_path, codec=codec, write_logfile=False);
MPYWriteVideoFile(video_clip, output_path, codec=codec, write_logfile=False);
else:
MPYWriteVideoFile(video_clip, output_path, codec=codec, write_logfile=False, bitrate=bitrate);
# video_clip.write_videofile(output_path, codec=codec, write_logfile=False, bitrate=bitrate);
if (return_vid):
return Video(output_path);
else:
return True;
@staticmethod
def CreateByStackingVideos(video_objects=None, video_paths=None, output_path=None, audio = None, concatdim = 0, force_recompute=True, **kwargs):
assert output_path, "MUST PROVIDE OUTPUT PATH FOR VIDEO"
if(not force_recompute):
if(os.path.isfile(output_path)):
return Video(path=output_path);
matchdim = (concatdim+1)%2;
vids=[];
if(video_objects is not None):
vids=video_objects;
if(video_paths is not None):
for vp in video_paths:
vids.append(Video(path=video_paths));
output_path=output_path.encode(sys.getfilesystemencoding()).strip();
make_sure_dir_exists(output_path);
basevid = vids[0];
if(audio is None):
audio = basevid.audio;
sampling_rate = basevid.sampling_rate;
tempfilepath = get_temp_file_path(final_file_path=output_path, temp_dir_path=vids[0].getTempDir());
basevid.openVideoWriter(output_file_path=tempfilepath, fps=sampling_rate);
duration = basevid.getDuration();
nsamples = sampling_rate*duration;
old_frame_time = truediv(1.0,sampling_rate);
#frame_start_times = np.linspace(self.getStartTime(),self.getEndTime(),num=nsamples,endpoint=False);
frame_start_times = np.linspace(0,duration,num=nsamples,endpoint=False);
#frame_index_floats = frame_start_times/old_frame_time;
frame_index_floats = frame_start_times*sampling_rate;
for nf in range(len(frame_index_floats)):
frameind = frame_index_floats[nf];
newframe = basevid.getFrame(frameind);
for vn in range(1,len(vids)):
addpart = vids[vn].getFrame(frameind);
partsize = np.asarray(addpart.shape)[:];
cumulsize = np.asarray(newframe.shape)[:];
if(partsize[matchdim]!=cumulsize[matchdim]):
sz = partsize[:];
sz[matchdim]=cumulsize[matchdim];
addpart = sp.misc.imresize(addpart, size=sz);
newframe = np.concatenate((newframe, addpart), concatdim);
basevid.writeFrame(newframe);
basevid.closeVideoWriter();
rvid = Video.CreateFromVideoAndAudio(video_path=tempfilepath, audio_object=audio, output_path=output_path, **kwargs);
os.remove(tempfilepath);
return rvid;
@staticmethod
def CreateFromVideoAndAudioPaths(video_path, audio_path, output_path, return_vid = True, **kwargs):
return Video.CreateFromVideoAndAudio(video_path=video_path, audio_path=audio_path, output_path=output_path,return_vid=return_vid, **kwargs);
@staticmethod
def CreateFromVideoAndAudioObjects(video, audio, output_path, clip_to_video_length=True, return_vid = True, **kwargs):
return Video.CreateFromVideoAndAudio(video_object=video, audio_object=audio, output_path=output_path, clip_to_video_length=clip_to_video_length, return_vid = return_vid, **kwargs);
def _getDefaultPeakPickingTimeParams(self, **kwargs):
single_frame = np.true_divide(1.0, self._getFrameRate());
time_params = dict(
pre_max_time=2.0 * single_frame,
post_max_time=2.0 * single_frame,
pre_avg_time=5.0 * single_frame,
post_avg_time=5.0 * single_frame,
wait_time=2.0 * single_frame,
delta=0.015,
)
time_params.update(kwargs);
return time_params;
# ###################
# FEATURE
def getFrameIndexes(self, force_recompute=False):
feature_name = 'frame_indexes';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = dict();
duration = self.getDuration();
nsamples = self.sampling_rate * duration;
frame_start_times = np.linspace(0, duration, num=nsamples, endpoint=False);
value = frame_start_times * self.sampling_rate;
self.setFeature(name=feature_name, value=value, params=params);
return self.getFeature(feature_name);
def getFeaturesList(self):
return super(Video, self).getFeaturesList()+self.audio.getFeaturesList();
def getVideoFeaturesList(self):
return super(Video, self).getFeaturesList();
def getFeatureFunctionsList(self):
return super(Video, self).getFeatureFunctionsList()+self.audio.getFeatureFunctionsList();
def getFeatureSourceType(self, name):
"""returns whether video or audio feature, instead of parent version which returns true or false"""
vidhas = super(Video, self).hasFeature(name=name);
if(vidhas):
return 'video';
audiohas = self.audio.hasFeature(name=name);
if(audiohas):
return 'audio';
return None;
def getFeature(self, name, force_recompute=False, **kwargs):
"""Returns None if feature is not already computed, and feature name is not implemented and registered with FEATURE_FUNCS"""
params = kwargs;
assert(not kwargs.get('params')), "STILL TRYING TO USE PARAMS INSTEAD OF KWARGS. FIX THIS";
ftry = super(Video, self).getFeature(name=name, force_recompute=force_recompute, **kwargs);
if(ftry is not None):
return ftry;
else:
return self.audio.getFeature(name=name, force_recompute=force_recompute, **kwargs);
Video.FEATURE_FUNCS['frame_indexes']=Video.getFrameIndexes;
from . import Video_CV;
if(Video_CV.USING_OPENCV):
Video.FEATURE_FUNCS.update(Video_CV.FEATURE_FUNCS);
Video.USING_OPENCV = Video_CV.USING_OPENCV;
Video.localRhythmicSaliencyFunction = Video_CV.localRhythmicSaliencyFunction
Video.visualBeatFunction = Video_CV.visualBeatFunction
Video.cvGetGrayFrame = Video_CV.cvGetGrayFrame;
Video.getImageFromFrameGray = Video_CV.getImageFromFrameGray;
Video.plotVisualBeats = Video_CV.plotVisualBeats;
Video.loadFlowFeatures = Video_CV.loadFlowFeatures;
Video.getVisualBeats = Video_CV.getVisualBeats
Video.getLocalRhythmicSaliency = Video_CV.getLocalRhythmicSaliency;
Video.getDirectogram = Video_CV.getDirectogram;
Video.getDirectogramPowers = Video_CV.getDirectogramPowers;
Video.getVisibleImpactEnvelope = Video_CV.getVisibleImpactEnvelope;
Video.getVisibleImpactEnvelopePowers = Video_CV.getVisibleImpactEnvelopePowers;
Video.getVisibleImpacts = Video_CV.getVisibleImpacts;
Video.getVisualBeats = Video_CV.getVisualBeats;
# Video.getBackwardVisualBeats = Video_CV.getBackwardVisualBeats;
# Video.getForwardVisualBeats = Video_CV.getForwardVisualBeats;
Video.getBackwardVisibleImpactEnvelope = Video_CV.getBackwardVisibleImpactEnvelope;
Video.getBothWayVisibleImpactEnvelope = Video_CV.getBothWayVisibleImpactEnvelope;
Video.getForwardVisibleImpactEnvelope = Video_CV.getForwardVisibleImpactEnvelope;
Video.getDirectionalFlux = Video_CV.getDirectionalFlux;
Video.getVisualTempogram = Video_CV.getVisualTempogram;
Video.getCutEvents = Video_CV.getCutEvents;
Video.computeImpactEnvelope = Video_CV.computeImpactEnvelope;
Video.computeDirectogramPowers = Video_CV.computeDirectogramPowers;
Video.visualBeatsFromEvents = Video_CV.visualBeatsFromEvents;
Video.plotVisualBeats = Video_CV.plotVisualBeats;
Video.plotImpactEnvelope = Video_CV.plotImpactEnvelope;
Video.plotVisibleImpacts = Video_CV.plotVisibleImpacts;
Video.getVisualBeatSequences = Video_CV.getVisualBeatSequences;
Video.printVisualBeatSequences = Video_CV.printVisualBeatSequences;
Video.getVisualBeatTimes = Video_CV.getVisualBeatTimes;
Video.findAccidentalDanceSequences = Video_CV.findAccidentalDanceSequences;
Video.plotEvents = Video_CV.plotEvents;
# Video. = Video_CV.
# Video. = Video_CV.
# Video. = Video_CV.
# Video. = Video_CV.
# Video. = Video_CV.
# Video. = Video_CV.
else:
AWARN("Was not able to add functions that use OpenCV! Check OpenCV instalation and try again?");
from .vbgui import BeatGUI
if(BeatGUI.VIEWER_INSTALLED):
def getEvents(self, **kwargs):
time_params = self._getDefaultPeakPickingTimeParams();
time_params.update(kwargs)
vbeats = self.getFeature('visual_beats', force_recompute=True, **time_params);
return vbeats;
def getEventList(self, **kwargs):
events = self.getEvents(**kwargs);
return EventList(events=events);
def runBeatGUIOnAudio(self):
audio = self.getAudio();
self.gui.run(local_saliency=audio.getLocalRhythmicSaliency(), frame_rate = audio._getFrameRate(), eventlist = audio.getEventList());
def runGUI(self, local_saliency=None, frame_rate = None, eventlist = 'default', frame_offset=None):
self.gui.run(local_saliency=local_saliency, frame_rate=frame_rate, eventlist=eventlist, frame_offset=frame_offset);
Video._getGui = BeatGUI.media_GUI_func;
Video.runGUI = runGUI;
Video.getEvents = getEvents;
Video.getEventList = getEventList;
Video.runBeatGUIOnAudio = runBeatGUIOnAudio;
else:
AWARN("BeatGUI not installed");
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/VideoClip.py
================================================
from .Video import *
from .AudioClip import *
class VideoClip(Video):
"""VideoClip (class): A segment of a video, and a bunch of convenience functions to go with it.
Attributes:
start: The name of the video
end: The framerate of the video
"""
def AOBJECT_TYPE(self):
return 'VideoClip';
def __init__(self, video=None, start=None, end=None, clip_to_frame=True, path=None):
"""If a video is provided, we don't want to reload the video from disk. If a path is provided, we have no choice.
"""
assert(not (video and path)), "provided both video object and path to VideoClip constructor."
assert((start is not None) and (end is not None)), "must provide start time and end time to AudioClip constructor"
Video.__init__(self, path = path);
self.initializeBlank();
self.start = start;
self.end = end;
if(video):
self.setPath(video.getPath());
self.reader = video.reader;
#self.writer = None; #this only comes up if we write later
self.sampling_rate=video.sampling_rate;
self.num_frames_total=video.num_frames_total;
time_per_frame=truediv(1.0,self.sampling_rate);
if(clip_to_frame):
self.start=time_per_frame*math.floor(truediv(self.start,time_per_frame));
self.end=time_per_frame*math.floor(truediv(self.end,time_per_frame));
if(video.name):
self.name = video.name+"_{}_{}".format(start,end);
self.audio = video.audio.AudioClip(start=start, end=end);
else:
assert(False),"must provide video to VideoClip init"
def initializeBlank(self):
Video.initializeBlank(self);
self.start = None;
self.end = None;
# def readFrameBasic(self, i):
# return Video.getFrame(self, float(i)+self.start);
def getFrameLinearInterp(self, f):
return Video.getFrameLinearInterp(self, float(f)+self.start*self.sampling_rate);
def getFrame(self, f):
return Video.getFrame(self, float(f)+self.start*self.sampling_rate);
def getDuration(self, round_to_frames=False):
if(not round_to_frames):
return self.end-self.start;
else:
return truediv(self.n_frames(), self.sampling_rate);
def n_frames(self):
#return self.getLastFrameIndex()-self.getFirstFrameIndex();
return math.ceil((self.end-self.start)*self.sampling_rate);
# def getFirstFrameIndex(self):
# return math.floor(self.start*self.sampling_rate);
#
# def getLastFrameIndex(self):
# #I think floor is still right here, because the times mark beginnings of frames
# return math.floor(self.end*self.sampling_rate);
def getStartTime(self):
return self.start;
def getEndTime(self):
return self.end;
def getMPYClip(self, get_audio=True):
return mpy.VideoFileClip(self.getPath(), audio=get_audio).subclip(self.getStartTime(), self.getEndTime());
def play(self):
if(ISNOTEBOOK):
print("Playing video:")
IPython.display.display(self.getMPYClip().ipython_display(fps=self.sampling_rate, maxduration=self.getDuration()+1));
else:
print("HOW TO PLAY VIDEO? NOT A NOTEBOOK.")
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/VideoSource.py
================================================
from .VisBeatImports import *
from .Video import *
from .AFileManager import AFileManager
try:
import youtube_dl
except ImportError:
AWARN('You need to install youtube-dl to use parts of VideoSource class that pull from YouTube. Try running:\npip install youtube-dl')
def safe_file_name(input_string):
return ''.join([i if ord(i) < 128 else '_' for i in input_string]);
class VideoSource(AFileManager):
"""Video (class): A video, and a bunch of convenience functions to go with it.
Attributes:
name: The name of the video
sampling_rate: The framerate of the video
audio: an Audio object, the audio for the video
"""
VIDEO_TEMP_DIR = './'
def getJSONName(self):
return self.AOBJECT_TYPE()+".json";
def AOBJECT_TYPE(self):
return 'VideoSource';
def __init__(self, path, name=None, source_location=None, VideoClass = None, **kwargs):
"""
:param path: either the path to a json, or the path to a directory containing 'VideoSource.json',
or the path to a directory where said json should be created.
:param name: name for video
:param source_location: can be a path to a video file, or a youtube source from which to pull a video file
:param VideoClass: visbeat.Video by default, but can be changed if you are using a custom subclass.
Must be a subclass of visbeat.Video, and must be constructable with VideoClass(path)
"""
AFileManager.__init__(self, path=path);
if(VideoClass is not None):
self.VideoClass = VideoClass;
self.setSource(source_location=source_location, assert_valid=None, **kwargs);
# if (self.name is None):
# self.name = os.path.splitext(os.path.basename(path))[0]
@staticmethod
def NewVideoSource(destination, name, source_location=None, VideoClass = None, **kwargs):
vsdir = os.path.join(destination, name+os.sep);
make_sure_dir_exists(vsdir);
print(("Video source at {}".format(vsdir)));
return VideoSource(path=vsdir, name=name, source_location=source_location, **kwargs);
def initializeBlank(self):
AFileManager.initializeBlank(self);
self.source_location = None;
self.name = None;
self.video_file_name = None;
self.title_safe = None;
self.youtube_info_dict = None;
self.versions_info = {};
#
self.VideoClass = Video;
def toDictionary(self):
d = AFileManager.toDictionary(self);
d['source_location']=self.source_location;
if(self.name):
d['name']=self.name;
if(self.video_file_name):
d['video_file_name']=self.video_file_name;
if(self.title_safe):
d['title_safe']=self.title_safe;
if(self.youtube_info_dict):
d['youtube_info_dict']=self.youtube_info_dict;
if(self.versions_info):
d['versions_info']=self.versions_info;
return d;
def initFromDictionary(self, d):
AFileManager.initFromDictionary(self, d);
self.source_location = d['source_location'];
self.name = d.get('name');
self.video_file_name = d.get('video_file_name');
self.title_safe = d.get('title_safe');
self.youtube_info_dict = d.get('youtube_info_dict');
self.versions_info=d.get('versions_info');
if(self.versions_info is None):
self.versions_info = {};
#do class specific inits with d;
def initWithPath(self, path=None, clear_temp=None):
AFileManager.initWithPath(self, path=path, clear_temp=clear_temp);
self.setDir('versions', pathstring(self.getDirectoryPath() + os.sep + "Versions" + os.sep));
self.setDir('warps', pathstring(self.getDir('versions')+ os.sep + "Warps" + os.sep));
# self.setDir('midi', pathstring(self.getDirectoryPath() + os.sep + "MIDI" + os.sep));
# self.setDir('music', pathstring(self.getDirectoryPath() + os.sep + "Music" + os.sep));
self.setFeaturesDir();
def setFeaturesDir(self, features_dir=None):
if (features_dir is None):
self.setDir('features', pathstring(self.getDir('data') + os.sep + "Features" + os.sep))
else:
self.setDir('features', features_dir);
def getName(self):
"""
Gets name if set. If not set, returns file name without extension.
:return:
"""
if(self.name is not None):
return self.name;
elif(self.video_file_name is not None):
return os.path.splitext(self.video_file_name)[0];
else:
return None;
###################
@staticmethod
def _versionLabelString(version_label=None):
if (version_label and version_label != 'Full'):
return str(version_label);
else:
return 'Full';
@staticmethod
def _versionGroupString(version_group=None):
if (version_group is None):
version_group = 'Original';
return version_group;
@staticmethod
def _getVersionLabelDirName(version_label=None):
if(isinstance(version_label, int)):
return ("maxheight_{}".format(version_label));
else:
return "Full";
def getVersionPath(self, version_label=None, version_group=None):
"""
Get path to version of video (path to video file). Return None if version has not been added.
:param version_label:
:return:
"""
return self.getVersionInfo(version_label=version_label, version_group=version_group, info_label='path');
def getDirForVersion(self, version_label=None, version_group=None):
vdir = self.getDir('versions') + os.sep + self._versionGroupString(version_group) + os.sep + self._getVersionLabelDirName(version_label)+os.sep;
return vdir;
def getVersionInfo(self, version_label, version_group=None, info_label=None):
"""
Get info about a version of the video
:param version_label:
:param info_label:
:return:
"""
assert(info_label is not None), "should provide info_label to getVersionInfo()"
d = self.getVersionDictionary(version_label=version_label, version_group=version_group);
if(d is not None):
return d.get(info_label);
else:
return None;
def getVersionDictionary(self, version_label=None, version_group=None):
if(version_group is None):
version_group='Original';
vis_d=self.versions_info.get(version_group);
if(vis_d is not None):
return vis_d.get(VideoSource._versionLabelString(version_label));
else:
return None;
def setVersionDictionary(self, version_label=None, version_group=None, d=None):
if(version_group is None):
version_group='Original';
vis_d=self.versions_info.get(version_group);
if(vis_d is None):
self.versions_info[version_group]={};
vis_d=self.versions_info.get(version_group);
vis_d[VideoSource._versionLabelString(version_label)]=d;
return;
def setVersionInfo(self, version_label=None, version_group=None, video_path=None, **kwargs):
version_dict = self.getVersionDictionary(version_label=version_label, version_group=version_group);
if(version_dict is None):
self.setVersionDictionary(version_label=version_label, version_group=version_group, d={});
version_dict = self.getVersionDictionary(version_label=version_label, version_group=version_group);
if(video_path is not None):
version_dict.update({'path':str(pathstring(video_path))});
if(kwargs is not None):
version_dict.update(kwargs);
####################
def hardSave(self):
if (os.path.isfile(self.getJSONPath())):
os.rename(self.getJSONPath(), self.getDir('backup') + os.sep + self.AOBJECT_TYPE() + ".json");
self.writeToJSON(self.getJSONPath());
def save(self):
if (self.getInfo('block_writing_to_json')):
return;
self.hardSave();
####################
# def removeAllVersionFiles(self, asset_manager=None):
# AWARN("This should wipe the directory, ya?")
# # AWARN("Remove all version files does not remove features and vis images! This still needs to be implemented!")
# # for vtype in self.versions_info:
# # for v in self.versions_info.get(vtype):
# # # print(self.versions_info.get(vtype).get(v));
# # vpath = self.versions_info.get(vtype).get(v).get('path');
# # assert(vpath), "version {} did not have path".format(vtype)
# # print(vpath)
# # if(os.path.isfile(vpath)):
# # print("REMOVING {}".format(vpath))
# # os.remove(vpath);
# # if(asset_manager is not None and vtype=='Original'):
# # if(v=='Full'):
# # rs=None;
# # else:
# # rs = int(v);
# # asset_manager.removeFeaturesForLinkRes(link=self, max_height=rs);
# # return True;
# def removeFeaturesForRes(self, max_height=None):
# v = self.getVersionVideo(name=link.name, max_height = max_height);
# v.clearFeatureFiles(features_to_clear=['all', 'each']);
#
#
# def removeFeatureFilesForVideo(self, video, features_to_remove=None):
# if(features_to_remove is None):
# return;
# if(not isinstance(features_to_remove, list)):
# features_to_remove=[features_to_remove];
# for f in features_to_remove:
# self.removeFeatureFileForVideo(video=video, feature_name=f);
#
#
# def removeFeatureFileForVideo(self, video, feature_name):
# AWARN("still need to implement removeFeatureFileForVideo. Should just empty corresponding directory. Also implement 'each' that removes all.")
# # if(feature_name=='each'):
# # return self.removeFeatureFilesForVideo(video=video, features_to_remove=video.getFeatureFunctionsList());
# # max_height = video.getInfo(label='max_height');
# # link = self.hasLinkWithName(name=video.name);
# # source_type=video.getFeatureSohahurceType(feature_name);
# # ipath = self.getFeaturePathForLink(feature_name=feature_name, link=link, max_height=max_height, source_type=source_type);
# # if(os.path.isfile(ipath)):
# # print("REMOVING {}".format(ipath));
# # os.remove(ipath);
#
# def removeFeatureFiles(self, video, feature_name):
# if(feature_name=='each'):
# AWARN("IMPLEMENT DELETE FEATURE DIR AND CREATE CLEAN");
# version_label = video.getInfo(label='version_label');
# AWARN("IMPLEMENT DELETE {} FOR VERSION_LABEL {}".format(feature_name, version_label));
# # ipath = self.getFeaturePathForLink(feature_name=feature_name, link=link, max_height=max_height, source_type=source_type);
# # if(os.path.isfile(ipath)):
# # print("REMOVING {}".format(ipath));
# # os.remove(ipath);
#
# def removeVersion(self, version_label=None, remove_files=True):
# AWARN("Remove Resolution does not remove features! This still needs to be implemented!")
# for vtype in self.versions_info:
# v = self.versions_info.get(vtype).get(VideoSource._versionLabelString(version_label));
# if(v is not None):
# vpath = v.get('path');
# if(os.path.isfile(vpath)):
# print("REMOVING {}".format(vpath))
# os.remove(vpath);
# self.versions_info.get(vtype).pop(VideoSource._versionLabelString(version_label), None);
# return True;
#
# def copyResolutionTo(self, version_label=None, output_dir=None):
# assert(output_dir)
# for vtype in self.versions_info:
# v = self.versions_info.get(vtype).get(VideoSource._versionLabelString(version_label));
# if(v is not None):
# vpath = v.get('path');
# if(os.path.isfile(vpath)):
# opath = os.path.join(output_dir,(vtype+'_'+VideoSource._versionLabelString(version_label)+self.video_file_name));
# print("COPYING {} to {}".format(vpath,opath));
# shutil.copy2(vpath, opath);
# return True;
#
# def copyAssetsTo(self, output_dir=None, asset_manager=None):
# for vtype in self.versions_info:
# output_type_dir=pathstring(output_dir+os.sep+vtype+os.sep);
# make_sure_dir_exists(output_type_dir);
# for v in self.versions_info.get(vtype):
# output_a_dir = pathstring(output_type_dir+os.sep+v+os.sep);
# make_sure_dir_exists(output_a_dir);
# # print(self.versions_info.get(vtype).get(v));
# vpath = self.versions_info.get(vtype).get(v).get('path');
# if(vpath and os.path.isfile(vpath)):
# print("copying {} to {}".format(vpath, output_a_dir));
# shutil.copy2(vpath, output_a_dir);
# if(asset_manager is not None and vtype=='Original'):
# if(v=='Full'):
# rs=None;
# else:
# rs = int(v);
# asset_manager.saveFeaturesForVersion(version_label=rs, output_dir = output_a_dir);
def getWarpsDir(self, version_label=None):
warpdir = self.getDir('warps');
resdir = self._getVersionLabelDirName(version_label=version_label);
rdir = pathstring(warpdir + os.sep + resdir + os.sep);
make_sure_path_exists(rdir);
return rdir;
# ############################# FROM ASSETMANAGER #######################
def getVersion(self, max_height=None, get_if_missing=True, load_features=True, **kwargs):
"""
Gets a version of the video with maximum height max_height. This function contains logic to decide whether/when
to pull the video. Use pullVersion to force pulling that overwrite's existing assets.
:param max_height:
:param get_if_missing:
:return:
"""
vpath = self.getVersionPath(version_label=max_height);
if(vpath and os.path.isfile(vpath)):
num_frames_total = self.getVersionInfo(version_label=max_height, info_label='num_frames_total');
if(num_frames_total):
v = self.RegisteredVideo(path=vpath, version_label = max_height, num_frames_total=num_frames_total, load_features=True);
else:
v = self.RegisteredVideo(path=vpath, version_label = max_height, load_features=True);
self.setVersionInfo(version_label=max_height, num_frames_total=v.num_frames_total);
self.save();
return v;
else:
if(get_if_missing):
vpath = self.pullVersion(max_height=max_height, **kwargs);
if(vpath):
num_frames_total = self.getVersionInfo(version_label=max_height, info_label='num_frames_total');
if(num_frames_total):
v = self.RegisteredVideo(path=vpath, version_label = max_height, num_frames_total=num_frames_total, load_features=True);
else:
v = self.RegisteredVideo(path=vpath, version_label=max_height, load_features=True);
self.setVersionInfo(version_label=max_height, num_frames_total=v.num_frames_total);
self.save();
else:
AWARN("COULDNT GET VIDEO FROM {}".format(self.source_location));
else:
AWARN("COULDNT FIND VIDEO LOCALLY");
return;
return v;
def saveFeaturesForVideo(self, video, features_to_save=None, output_dir=None, overwrite=True):
if(features_to_save is None):
return;
if(not isinstance(features_to_save, list)):
features_to_save=[features_to_save];
for f in features_to_save:
self.saveFeatureForVideo(video=video, feature_name=f, output_dir=output_dir, overwrite=overwrite);
def saveFeatureForVideo(self, video, feature_name, output_dir=None, overwrite=True):
if(feature_name=='each'):
return self.saveFeaturesForVideo(video=video, features_to_save=video.getFeaturesList(), output_dir=output_dir, overwrite=overwrite);
version_label = video.getInfo(label='version_label');
source_type = video.getFeatureSourceType(feature_name);
opath = self.getFeaturePath(feature_name=feature_name, version_label=version_label, source_type=source_type, output_dir=output_dir);
make_sure_dir_exists(opath);
if(not os.path.isfile(opath) or overwrite):
if(feature_name=='all'):
video.saveFeatures(path=opath);
return;
else:
vfeature = video.getFeature(name=feature_name, force_recompute=False);
if(vfeature is not None):
video.saveFeature(name=feature_name, path=opath);
def loadFeaturesForVideo(self, video, features_to_load=None):
if(features_to_load is None):
return;
if(not isinstance(features_to_load, list)):
features_to_load=[features_to_load];
for f in features_to_load:
self.loadFeatureForVideo(video=video, feature_name=f);
def loadFeatureForVideo(self, video, feature_name):
if(feature_name=='each'):
return self.loadFeaturesForVideo(video=video, features_to_load=video.getFeatureFunctionsList());
version_label = video.getInfo(label='version_label');
source_type=video.getFeatureSourceType(feature_name);
ipath = self.getFeaturePath(feature_name=feature_name, version_label=version_label, source_type=source_type);
if(os.path.isfile(ipath)):
if(feature_name=='all'):
video.loadFeatures(path=ipath);
else:
video.loadFeature(name=feature_name, path=ipath);
def getFeaturePath(self, feature_name=None, source_type=None, version_label=None, output_dir=None):
assert(feature_name is not None), 'must provide name of feature VideoSource.getFeaturePath'
feature_dir = self.getFeatureDir(feature_name=feature_name, source_type = source_type);
fileext = '.pkl';
if(output_dir is None):
version = self._getVersionLabelDirName(version_label=version_label)+os.sep;
output_dir = pathstring(feature_dir);
outname = self.getName();
if(outname is None):
outname = 'version';
outname = outname+'_'+self._getVersionLabelDirName(version_label=version_label)+fileext;
opath = os.path.join(output_dir, outname);
return opath;
def getFeatureDir(self, feature_name=None, source_type=None):
if(source_type is None):
source_type='video'; # could be 'audio'
# AWARN("getDir('features')= {}\nsource_type= {}\nfeature_name= {}".format(self.getDir('features'), source_type, feature_name))
ftypedir = pathstring(self.getDir('features') + os.sep + source_type + os.sep + feature_name + os.sep);
return ftypedir;
def saveFeaturesForVersion(self, version_label=None, output_dir=None):
assert(output_dir), 'must provide output dir';
v = self.getVersion(max_height=version_label);
v.save(features_to_save='all', output_dir=output_dir);
def RegisteredVideo(self, path=None, version_label=None, version_group = None, num_frames_total=None, load_features = True):
v = self.VideoClass(path=path, name=self.getName()+'_'+self._versionLabelString(version_label=version_label), num_frames_total=num_frames_total);
# v = Video(path=path, name=self.getName()+'_'+self._versionLabelString(version_label=version_label), num_frames_total=num_frames_total);
self.RegisterVideo(video=v, version_label=version_label, load_features=load_features);
return v;
def RegisterVideo(self, video, version_label = None, version_group = None, load_features=True):
def videosave(vidob, features_to_save='all', output_dir=None, overwrite=True):
self.saveFeaturesForVideo(video=vidob, features_to_save=features_to_save, output_dir=output_dir, overwrite=overwrite);
#self.save
def videoload(vidob, features_to_load='all', input_dir=None):
self.loadFeaturesForVideo(video=vidob, features_to_load=features_to_load);
def videoclear(vidob, features_to_clear='all'):
self.removeFeatureFilesForVideo(video=vidob, features_to_remove=features_to_clear);
video.setInfo(label='version_label', value=version_label);
video.setInfo(label='version_group', value=version_group);
video.setInfo(label='video_file_name', value=os.path.split(video.getPath())[1]);
# if(version_group is None):
video.save_func = videosave;
video.load_func = videoload;
video.clear_feature_files_func = videoclear;
video.source=self;
if(load_features is True and hasattr(video, "loadFlowFeatures")):
video.loadFlowFeatures();
elif(load_features is not None and (isinstance(load_features, list) or isinstance(load_features, str))):
video.load(features_to_load = load_features);
# return video;
def addVersion(self, path, version_label=None, version_group=None):
v = self.RegisteredVideo(path=path, version_label=version_label);
video_dict=v.getVersionInfo();
video_dict.update(dict(version_group=version_group));
self.setVersionInfo(video_path = path, version_label=version_label, **video_dict);
self.save();
def addVersionToVideo(self, video, new_video, version_label=None, version_group=None):
version_label = video.getInfo(label='version_label');
video_dict = new_video.getVersionInfo();
if ((version_group is not None) or (video_dict.get('version_group') is None)):
video_dict.update(dict(version_group=version_group));
self.setVersionInfo(video_path=new_video.getPath(), version_label=version_label, **video_dict);
self.save();
def setSource(self, source_location=None, assert_valid=True, **kwargs):
"""
Sets the source of this VideoSource. If file, copies the file to VideoSource directory as "Original" version,
unless copy argument is set to false.
:param source_location: either path to file, or youtube url
:param kwargs: see setSourceYoutube or setSourceFile for options
:return:
"""
if(source_location):
if(os.path.isfile(source_location)):
self.setSourceFile(path = source_location, **kwargs);
else:
self.setSourceYoutube(url=source_location, **kwargs);
return;
elif(kwargs.get('video_path')):
self.setSourceFile(**kwargs);
return;
elif(kwargs.get('url')):
self.setSourceYoutube(**kwargs);
return;
else:
if(assert_valid):
assert(False),'No valid source location provided to setSource.'
def setSourceYoutube(self, url=None, max_height=None, pull_fullres=True, **kwargs):
self.source_location=url;
self.setInfo(label='source_type', value='youtube');
if(pull_fullres):
self.pullYoutubeVideo(max_height=max_height, **kwargs);
self.save();
def setSourceFile(self, path=None, copy=True, **kwargs):
# assert(os.path.isfile(path)), 'Tried to set source but no file exists at {}'.format(path);
if(not os.path.isfile(path)):
return None;
self.video_file_name = os.path.basename(path);
if(not copy):
self.source_location = path;
self.setInfo(label='source_type', value='file_address');
return path;
# output_dir = self.getDir('versions') + os.sep + 'Original' + os.sep;
output_dir = self.getDirForVersion(version_label=None, version_group='Source');
make_sure_dir_exists(output_dir);
output_path = os.path.join(output_dir, self.video_file_name);
# max_height = kwargs.get('max_height');
# if(max_height is not None):
# # print('max_height was {}'.format(max_height));
# original = self.VideoClass(path=path);
# original.writeResolutionCopyFFMPEG(path=output_path, max_height=kwargs.get('max_height'));
# else:
# shutil.copy2(path, output_path);
shutil.copy2(path, output_path);
# self.addVersion(path=output_path, version_label='Original');
self.addVersion(path=output_path, version_label='Full');
self.setInfo(label='source_type', value='file');
self.source_location = output_path;
self.save();
return output_path;
def pullVersion(self, max_height=None, **kwargs):
# assert(self.source_location is not None), "Could not pull version; source location is None."
if(self.source_location is None):
return None;
source_type = self.getInfo('source_type');
if(source_type=='youtube'):
return self.pullYoutubeVideo(max_height=max_height, **kwargs);
else:
return self.pullFileVideo(max_height=max_height, **kwargs)
def pullFileVideo(self, max_height=None, force_recompute=None):
original_path = self.source_location;
assert(os.path.isfile(original_path)), 'SOURCE FILE {} IS MISSING'.format(self.source_location);
original = self.VideoClass(path=original_path);
output_dir = self.getDirForVersion(version_label=max_height, version_group=None);
make_sure_dir_exists(output_dir);
output_path = os.path.join(output_dir, self.video_file_name);
if(max_height is None):
if(os.path.normpath(original_path) != os.path.normpath(output_path)):
shutil.copy2(original_path, output_path);
else:
original.writeResolutionCopyFFMPEG(path=output_path, max_height=max_height);
self.addVersion(path=output_path, version_label=max_height);
return self.getVersionPath(version_label=max_height);
def pullYoutubeVideo(self, max_height=None, write_subtitles=False, save_youtube_info=False,
force_redownload=False):
"""
Downloads video from youtube with height<=max_height. Returns path to downloaded video.
:param max_height: maximum height of video to download
:param write_subtitles: whether to save the subtitles
:param save_youtube_info: whether to save the info json
:param force_redownload: whether to re-download if video exists
:return:
"""
assert (not self.getInfo('source_is_file')), "tried to call pullYoutubeVideo on file source {}.".format(self.getName());
old_vid_path = self.getVersionPath(version_label=max_height);
if ((not force_redownload) and old_vid_path and os.path.isfile(old_vid_path)):
AWARN("Old video version exists with path {}\nSkipping download...\n".format(old_vid_path))
return old_vid_path;
# output_dir = self.getDir('versions') + os.sep + self._getVersionLabelDirName(version_label=max_height) + os.sep;
output_dir = self.getDirForVersion(version_label=max_height, version_group=None);
make_sure_path_exists(output_dir);
if ((not force_redownload) and self.video_file_name is not None):
path_guess = os.path.join(output_dir, self.video_file_name);
if (os.path.isfile(path_guess)):
AWARN("found existing file {}\nSkipping download; set force_redownload=True to overwrite old version.".format(path_guess));
self.addVersion(path=path_guess, version_label=max_height);
return path_guess;
########Download From YouTube#########
vidpath_template = output_dir + '%(title)s-%(id)s' + '.%(ext)s';
ydl_opts = {
'format': 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best',
'outtmpl': vidpath_template,
}
if (write_subtitles):
ydl_opts['writesubtitles'] = True;
ydl_opts['subtitlesformat'] = '[srt]';
if (max_height):
ydl_opts['format'] = 'bestvideo[height<={}][ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best'.format(
max_height);
ydl_opts['writeinfojson'] = True;
info_dict = None;
AWARN("Downloading {} from {}...".format(self.getName(), self.source_location));
with youtube_dl.YoutubeDL(ydl_opts) as ydl:
info_dict = ydl.extract_info(self.source_location, download=True)
# video_source = info_dict.get("source", None)
# video_id = info_dict.get("id", None)
# video_title = info_dict.get('title', None)
usedtitle = info_dict['title'].replace('"', "'").replace('|', '_').replace(':', ' -').replace('/','_').replace('?', '');
usedfilename_withoutext = usedtitle + '-' + info_dict['id'];
usedfilename = usedfilename_withoutext + '.' + info_dict['ext'];
filepath = output_dir + os.sep + usedfilename;
filepathsafe = safe_file_name(filepath);
if (os.path.isfile(filepath)):
os.rename(filepath, filepathsafe);
else:
fpath = glob.glob(output_dir + os.sep + '*' + info_dict['id'] + '.mp4');
assert (len(fpath) == 1), "Wrong number of files for {}\nFound:\n{}".format(filepath, fpath);
os.rename(fpath[0], filepathsafe);
jpath = glob.glob(output_dir + os.sep + '*' + info_dict['id'] + '.info.json');
os.rename(jpath[0], safe_file_name(usedfilename_withoutext + '.info.json'));
print(("Saved to {}".format(filepathsafe)));
self.video_file_name = safe_file_name(usedfilename);
self.title_safe = safe_file_name(usedtitle);
if (save_youtube_info):
self.youtube_info_dict = info_dict;
# v = Video(path=filepathsafe);
# video_dict=v.toDictionary();
# self.setVersionInfo(video_path = filepathsafe, version_label=max_height, num_frames_total=v.num_frames_total, **video_dict);
# self.save();
self.addVersion(path=filepathsafe, version_label=max_height);
return self.getVersionPath(version_label=max_height);
# return youtube_link;
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/Video_CV.py
================================================
# from Video import *
from .Image import *
from .Event import *
from .VisualBeat import *
import librosa
FEATURE_FUNCS = {};
VIS_FUNCS = {};
VISVIDEO_FUNCS = {};
FLOW_LOG_EPSILON=1.0;
FLOW_UNIT_GAIN=10000.0;
HISTOGRAM_FRAMES_PER_BEAT = 2;
HISTOGRAM_DOWNSAMPLE_LEVELS = 3;
VB_UPSAMPLE_FACTOR = 1.0;
USING_OPENCV = Image.USING_OPENCV;
if(USING_OPENCV):
##########################################################################
# ################ THESE ARE THE FUNCTIONS TO OVERRIDE! ################ #
# ################ FOR CUSTOM SALIENCY METRICS! ################ #
##########################################################################
# You can modify them here. But I would suggest elsewhere in your code
# just setting Video.localRhythmicSaliencyFunction and
# Video.visualBeatFunction to whatever you want. See how they are assigned
# when Video_CV is included in Video.py. -Abe
def localRhythmicSaliencyFunction(self, **kwargs):
"""
Change to use different function for local saliency
"""
return self.getVisibleImpactEnvelope(**kwargs);
def visualBeatFunction(self, **kwargs):
"""
Change to use different default strategy for selecting visual beats
"""
# beat_params = dict(
# pre_max_time=0.2,
# post_max_time=0.2,
# pre_avg_time=0.2,
# post_avg_time=0.2,
# wait_time=0.1,
# )
beat_params = self._getDefaultPeakPickingTimeParams();
beat_params.update(kwargs);
return self.getVisibleImpacts(**beat_params);
# #################### This is how they are called #################### #
def getLocalRhythmicSaliency(self, force_recompute=False, **kwargs):
feature_name = 'local_rhythmic_saliency';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = dict();
params.update(kwargs);
params.update({'force_recompute':force_recompute});
result = self.localRhythmicSaliencyFunction(**params);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getVisualBeats(self, force_recompute=False, **kwargs):
feature_name = 'visual_beats';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
params.update({'force_recompute': force_recompute});
result = self.visualBeatFunction(**params);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
# ##################################################################### #
##########################################################################
# ###################################################################### #
##########################################################################
def cvGetGrayFrame(self, f):
colorframe = self.getFrame(f).astype(np.uint8);
return ocv.cvtColor(colorframe, ocv.COLOR_RGB2GRAY);
def getImageFromFrameGray(self, f):
frame = self.getImageFromFrame(f);
frame.RGB2Gray();
return frame;
def flow2row(ang, amp, bins, subdivs, n_shifts, density):
h, w = ang.shape[:2];
ncells = np.power(4, subdivs);
nperd = np.power(2, subdivs);
xw = w-n_shifts[1];
yw = h-n_shifts[0];
xcells = np.arange(nperd + 1, dtype=float);
ycells = np.arange(nperd + 1, dtype=float);
xcells = np.floor((xcells / (nperd)) * xw);
ycells = np.floor((ycells / (nperd)) * yw);
# print(n_shifts)
ahis = np.zeros([ncells * bins, n_shifts[0]*n_shifts[1]]);
for dy in range(n_shifts[0]):
for dx in range(n_shifts[1]):
ampwin = amp[dy:dy+yw,dx:dx+xw];
angwin = ang[dy:dy+yw,dx:dx+xw];
cell_counter = 0;
for x in range(nperd):
for y in range(nperd):
ystart=int(ycells[y]);
yend = int(ycells[y+1]);
xstart = int(xcells[x]);
xend = int(xcells[x + 1]);
angcell = angwin[ystart:yend, xstart:xend];
ampcell = ampwin[ystart:yend, xstart:xend];
cahis, cbinbounds = np.histogram(angcell.ravel(), bins=bins, range=(0, 2 * np.pi),
weights=ampcell.ravel(), density=density);
# print("ahis shape: {}\ncahis shape: {}\ndx, dy: {}, {}\ncell_counter: {}\nbins: {}\n".format(ahis.shape, cahis.shape, dx, dy, cell_counter, bins));
ahis[cell_counter * bins:(cell_counter + 1) * bins, dx+dy*n_shifts[0]] = cahis;
cell_counter = cell_counter + 1;
return ahis;
def getFlowFrame(self, frame_index):
prev_frame = self.cvGetGrayFrame(frame_index-1);
this_frame = self.vbGetGrayFrame(frame_index);
return cvDenseFlowFarneback(from_image=prev_frame, to_image=this_frame);
def getFlowFramePolar(self, frame_index):
"""
:param self:
:param frame_index:
:return: polar where polar[:,:,0] is amplitude, and polar[:,:,1] is angle
"""
flow = self.getFlowFrame(frame_index);
fx, fy = flow[:,:,0], flow[:,:,1];
polar = np.zeros(size(flow));
polar[:,:,0] = np.sqrt(fx * fx + fy * fy);
polar[:,:,1] = np.arctan2(fy, fx) + np.pi
return polar;
def computeDirectogramPowers(self, bins=None, dead_zone= 0.05, density=None, save_if_computed=True, noise_floor_percentile=None, **kwargs):
if(bins is None):
bins = 128;
if(noise_floor_percentile is None):
noise_floor_percentile = 20;
print(("Computing Flow Features with deadzone {}".format(dead_zone)))
signal_dim = 128;
m_histvals = np.zeros([signal_dim, self.n_frames(), 3]);
flow_averages = np.zeros([self.n_frames(), 1]);
sampling_rate=self.sampling_rate;
duration = self.getDuration();
nsamples = sampling_rate*duration;
# print(sampling_rate, duration)
frame_start_times = np.linspace(0,duration,num=int(nsamples),endpoint=False);
frame_index_floats = frame_start_times*self.sampling_rate;
lastframe = self.cvGetGrayFrame(frame_index_floats[0]);
start_timer=time.time();
last_timer=start_timer;
fcounter=0;
counter = 0;
for nf in range(len(frame_index_floats)):
nextframe= self.cvGetGrayFrame(frame_index_floats[nf]);
flow = cvDenseFlowFarneback(from_image=lastframe, to_image=nextframe);
h, w = flow.shape[:2];
fx, fy = flow[:,:,0], flow[:,:,1];
# if(filter_median):
# fx = fx-np.median(fx.ravel());
# fy = fy-np.median(fy.ravel());
# assert(False), "SHOULDNT BE FILTERING MEDIAN! VESTIGIAL CODE"
ang = np.arctan2(fy, fx) + np.pi
amp = np.sqrt(fx*fx+fy*fy);
winstarty = int(dead_zone*h);
winendy = h-winstarty;
winstartx = int(dead_zone*w);
winendx = w-winstartx;
angw = ang[winstarty:winendy, winstartx:winendx];
ampw = amp[winstarty:winendy, winstartx:winendx];
mask0 = (ampw>np.percentile(ampw,noise_floor_percentile)).astype(float);
ahis0, cbinbounds = np.histogram(angw.ravel(), bins=bins, range=(0, 2 * np.pi),
weights=mask0.ravel(), density=density);
ahis1, cbinbounds1 = np.histogram(angw.ravel(), bins=bins, range=(0, 2 * np.pi),
weights=ampw.ravel(), density=density);
ahis2, cbinbounds2 = np.histogram(angw.ravel(), bins=bins, range=(0, 2 * np.pi),
weights=np.power(ampw, 2).ravel(), density=density);
m_histvals[:, counter, 0] = ahis0;
m_histvals[:, counter, 1] = ahis1;
m_histvals[:, counter, 2] = ahis2;
lastframe=nextframe;
counter+=1;
fcounter+=1;
if(not (fcounter%50)):
if((time.time()-last_timer)>10):
last_timer=time.time();
print(("{}%% done after {} seconds...".format(100.0*truediv(fcounter,len(frame_index_floats)), last_timer-start_timer)));
params = dict( bins = bins,
deadzone=dead_zone,
density=density);
params.update(kwargs);
self.setFeature(name='directogram_powers', value=m_histvals, params=params);
if(save_if_computed):
self.save(features_to_save=['directogram_powers']);
return m_histvals;
######################### Other Features #############################
def getDirectogramPowers(self, force_recompute=False, **kwargs):
feature_name = 'directogram_powers';
if ((not self.hasFeature(feature_name)) or force_recompute):
flow_powers = self.computeDirectogramPowers(**kwargs);
return self.getFeature(feature_name);
# def getDirectogram(self, bins = None, weights=None, density=None, force_recompute=False, save_if_computed=True, **kwargs):
def getDirectogram(self, **kwargs):
feature_name = 'directogram';
force_recompute = kwargs.get('force_recompute');
if((not self.hasFeature(feature_name)) or force_recompute):
flow_powers = self.getFeature('directogram_powers');
fh = flow_powers[:,:,1];
self.setFeature(name='directogram', value=fh, params=kwargs);
return self.getFeature(feature_name);
def getVisualTempo(self, force_recompute=None, **kwargs):
feature_name = 'visual_tempo';
if ((not self.hasFeature(feature_name)) or force_recompute):
vbe = self.getFeature('local_rhythmic_saliency');
# assert librosa.__version__ == '0.7.1'
# result = librosa.beat.beat_track(onset_envelope=vbe, sr=self.sampling_rate, hop_length=1, **kwargs);
result = librosa.beat.beat_track(onset_envelope=vbe, sr=self.sampling_rate, hop_length=1, **kwargs);
self.setFeature(name=feature_name, value=result, params=kwargs);
return self.getFeature(feature_name);
def getVisualTempogram(self, window_length=None, force_recompute=None, norm_columns = None, **kwargs):
"""
:param self:
:param window_length: in seconds
:param force_recompute:
:param kwargs:
:return:
"""
feature_name = 'visual_tempogram';
if ((not self.hasFeature(feature_name)) or force_recompute):
if(window_length is None):
window_length = DEFAULT_TEMPOGRAM_WINDOW_SECONDS;
params = kwargs;
params.update({'force_recompute': force_recompute});
vbe = self.computeImpactEnvelope(cut_suppression_seconds = None);
onset_envelope = vbe;
win_length = int(round(window_length * self.sampling_rate));
sr = self.sampling_rate;
hop_length = 1;
center=kwargs.get('center');
if(center is None):
center=True;
window='hann'
norm=np.inf;
ac_window = librosa.filters.get_window(window, win_length, fftbins=True)
# Center the autocorrelation windows
n = len(onset_envelope)
if center:
onset_envelope = np.pad(onset_envelope, int(win_length // 2),
mode='linear_ramp', end_values=[0, 0])
# Carve onset envelope into frames
odf_frame = librosa.util.frame(onset_envelope,
frame_length=win_length,
hop_length=hop_length)
# Truncate to the length of the original signal
if center:
odf_frame = odf_frame[:, :n]
odf_frame = librosa.util.normalize(odf_frame,axis=0,norm=norm)
if(norm_columns is None):
norm_columns = True;
if(norm_columns):
# Window, autocorrelate, and normalize
result = librosa.util.normalize(
librosa.core.audio.autocorrelate(odf_frame * ac_window[:, np.newaxis], axis=0), norm=norm, axis=0);
else:
result = librosa.core.audio.autocorrelate(odf_frame * ac_window[:, np.newaxis], axis=0);
result = np.true_divide(result, np.max(result.ravel()));
tempo_bpms = librosa.tempo_frequencies(result.shape[0], hop_length=hop_length, sr=sr)
self.setFeature(name='tempogram_bpms', value=tempo_bpms);
###########
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getVisibleImpactEnvelope(self, force_recompute=False, **kwargs):
feature_name = 'impact_envelope';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
result = self.computeImpactEnvelope(forward=True, backward = False, **kwargs);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getForwardVisibleImpactEnvelope(self, force_recompute=False, **kwargs):
"""
Same as getVisibleImpactEnvelope by default.
:param self:
:param force_recompute:
:param kwargs:
:return:
"""
feature_name = 'forward_visual_impact_envelope';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
result = self.computeImpactEnvelope(forward=True, backward = False, **kwargs);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getBackwardVisibleImpactEnvelope(self, force_recompute=False, **kwargs):
feature_name = 'backward_visual_impact_envelope';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
result = self.computeImpactEnvelope(forward=False, backward = True, **kwargs);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getBothWayVisibleImpactEnvelope(self, force_recompute=False, **kwargs):
feature_name = 'both_way_visual_impact_envelope';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
result = self.computeImpactEnvelope(forward=True, backward = True, **kwargs);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getVisibleImpactEnvelopePowers(self, force_recompute=False, **kwargs):
feature_name = 'impact_envelope_powers';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
result0 = self.computeImpactEnvelope(power= 0, **params);
result = np.zeros([len(result0), 3]);
result[:, 0] = result0;
result[:, 1] = self.computeImpactEnvelope(power= 1, **params);
result[:, 2] = self.computeImpactEnvelope(power=2, **params);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getCutTimes(self):
return Event.ToStartTimes(self.getCutEvents());
def getCutEvents(self, force_recompute=False, **kwargs):
"""
Hacky estimate of cuts in a video
:param self:
:param force_recompute:
:param kwargs:
:return:
"""
feature_name = 'cut_events';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
powers = self.getFeature('directogram_powers');
im0 = powers[:, :, 0];
im2 = powers[:, :, 2];
imc = np.true_divide(im2, 0.05 + im0);
medsig = np.median(imc, 0);
medall = np.median(imc);
cutsig = np.true_divide(medsig, medall)
cut_detection_ratio = kwargs.get('cut_detection_ratio');
if(cut_detection_ratio is None):
cut_detection_ratio=CUT_DETECTION_RATIO;
clipsig = (cutsig > cut_detection_ratio).astype(float);
clip_floorsig = (cutsig > CUT_DETECTION_FLOOR).astype(float);
clipsig = np.multiply(clipsig, clip_floorsig);
einds = np.flatnonzero(clipsig);
etimes = einds * truediv(1.0, self.sampling_rate);
ev = Event.FromStartTimes(etimes, type='cut');
ev = self.visualBeatsFromEvents(ev);
evout = []
for e in range(len(ev)):
ev[e].setInfo('frame', einds[e]);
result = ev;
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def visualBeatsFromEvents(self, events):
def downsample_hist(sig, levels):
nperbin = np.power(2, levels);
rshp = sig.reshape(-1, nperbin);
return rshp.sum(axis=1);
if(self.hasFeature('impact_envelope')):
svbe=self.getFeature('impact_envelope');
else:
svbe=self.computeImpactEnvelope(cut_suppression_seconds = None);
flow_powers = self.getFeature('directogram_powers');
vis_tempogram = self.getFeature('visual_tempogram');
vbeats = [];
for e in events:
b = VisualBeat.FromEvent(e);
ei = int(round(b.start*self.sampling_rate*VB_UPSAMPLE_FACTOR));
b.weight = svbe[ei];
histsize = int(128/int(np.power(2,HISTOGRAM_DOWNSAMPLE_LEVELS)))
histslice = np.zeros([histsize, HISTOGRAM_FRAMES_PER_BEAT]);
histslice = np.squeeze(np.mean(histslice, 1));
b.flow_histogram = downsample_hist(sig=histslice, levels=HISTOGRAM_DOWNSAMPLE_LEVELS);
b.flow_histogram = np.divide(b.flow_histogram, np.sum(b.flow_histogram));
b.local_autocor = vis_tempogram[:,ei];
b.local_autocor = b.local_autocor/np.max(b.local_autocor);
b.sampling_rate=self.sampling_rate;
vbeats.append(b);
return vbeats;
def getVisualBeatTimes(self, **kwargs):
return Event.ToStartTimes(self.getVisualBeats(**kwargs));
def getDirectionalFlux(self,
f_sigma=None,
median_kernel=None,
power=None,
**kwargs):
"""
The visual impact complement of a spectral flux matrix
:param self:
:param f_sigma: sigma for the gaussian used to filter, using sp.ndimage.filters.gaussian_filter
:param median_kernel: used in sp.signal.medfilt
:param power: Which power of the flow to use. Usually use 1.
:param kwargs:
:return:
"""
def d_x(im):
d_im = np.zeros(im.shape);
d_im[:, 0] = im[:, 0];
for c in range(1, im.shape[1]):
d_im[:, c] = im[:, c] - im[:, c - 1];
return d_im;
if (f_sigma is None):
f_sigma = [5, 3];
if (median_kernel is None):
median_kernel = [3, 3];
if(power is None):
power = 1;
powers = self.getFeature('directogram_powers');
im = powers[:,:,power].copy();
if (f_sigma is not None):
im = sp.ndimage.filters.gaussian_filter(input=im, sigma=f_sigma, order=0);
im = sp.signal.medfilt(im, median_kernel);
return d_x(im);
def computeImpactEnvelope(self,
forward=True,
backward = False,
f_sigma=None,
median_kernel=None,
highpass_window_seconds= 0.8,
cut_percentile=99,
power=None,
crop = None,
normalize = True,
**kwargs):
"""
:param self:
:param forward: include impact going forward in time
:param backward: include impact going backward in time
:param f_sigma: sigma for the gaussian used to filter, using sp.ndimage.filters.gaussian_filter
:param median_kernel: used in sp.signal.medfilt
:param highpass_window_seconds: highpass window size in seconds
:param cut_percentile: percentile above which to consider cuts and to clip
:param power: Which power of the flow to use. Usually use 1.
:param crop:
:param kwargs:
:return:
"""
upsample_factor = kwargs.get('upsample_factor');
inputargs = dict(f_sigma=f_sigma,
median_kernel=median_kernel,
power=power,
crop = crop);
inputargs.update(kwargs);
im_d = self.getDirectionalFlux(**inputargs);
if(forward and backward):
im = np.fabs(im_d);
elif(forward):
im = -im_d;
im = np.clip(im, 0, None)
elif(backward):
im = im_d;
im = np.clip(im, 0, None)
else:
assert(False), "Must be at least one of either forward or backward."
vimpact = np.squeeze(np.mean(im, 0));
sampling_rate = self.sampling_rate;
if(upsample_factor is not None and (upsample_factor>1)):
newlen = upsample_factor * len(vimpact);
sampling_rate = upsample_factor*sampling_rate;
vimpact = sp.signal.resample(vimpact, newlen);
if(highpass_window_seconds):
order = kwargs.get('highpass_order');
if(order is None):
order = 5;
cutoff = truediv(1.0, highpass_window_seconds);
normal_cutoff = cutoff / (sampling_rate*0.5);
b, a = sp.signal.butter(order, normal_cutoff, btype='high', analog=False)
vimpact = sp.signal.filtfilt(b, a, vimpact);
normfactor = np.max(np.fabs(vimpact[:]));
if (cut_percentile is not None):
fx = np.fabs(vimpact);
pv = np.percentile(fx, cut_percentile);
pvlow = np.percentile(fx, cut_percentile-1);
normfactor = pv;
ptile = (vimpact > pv).astype(float);
pntile = (vimpact < -pv).astype(float);
pboth = ptile+pntile;
einds = np.flatnonzero(pboth);
lastind = -2;
for j in range(len(einds)):
if(einds[j]==(lastind+1)):
vimpact[einds[j]]=0;
else:
vimpact[einds[j]]=pvlow;
if(normalize):
vimpact = np.true_divide(vimpact, normfactor);
return vimpact;
# 0.8, cut_suppression_seconds = 0.4,
def computeImpactEnvelopeOld(self, f_sigma=None, median_kernel=None, highpass_window_seconds= 0.8, cut_percentile=99, power=None, crop = None, normalize=None, **kwargs):
"""
I believe this is the version of the function that I used in the original paper. Keeping it around for record.
:param self:
:param f_sigma:
:param median_kernel:
:param highpass_window_seconds:
:param cut_percentile:
:param power:
:param crop:
:param normalize:
:param kwargs:
:return:
"""
def d_x(im):
# d_im = np.zeros(im.shape, dtype=np.float128);
d_im = np.zeros(im.shape);
d_im[:, 0] = im[:, 0];
for c in range(1, im.shape[1]):
d_im[:, c] = im[:, c] - im[:, c - 1];
return d_im;
if (f_sigma is None):
f_sigma = [5, 3];
if (median_kernel is None):
median_kernel = [3, 3];
if(power is None):
power = 1;
upsample_factor = kwargs.get('upsample_factor');
powers = self.getFeature('directogram_powers');
print(("computing impact env with power {}".format(power)));
im = powers[:,:,power].copy();
if (f_sigma is not None):
im = sp.ndimage.filters.gaussian_filter(input=im, sigma=f_sigma, order=0);
im = sp.signal.medfilt(im, median_kernel);
im = -d_x(im);
im = np.clip(im, 0, None)
svbe = np.squeeze(np.mean(im, 0));
sampling_rate = self.sampling_rate;
if(upsample_factor is not None and (upsample_factor>1)):
newlen = upsample_factor * len(svbe);
sampling_rate = upsample_factor*sampling_rate;
svbe = sp.signal.resample(svbe, newlen);
if(highpass_window_seconds):
order = kwargs.get('highpass_order');
if(order is None):
order = 5;
cutoff = truediv(1.0, highpass_window_seconds);
normal_cutoff = cutoff / (sampling_rate*0.5);
b, a = sp.signal.butter(order, normal_cutoff, btype='high', analog=False)
svbe = sp.signal.filtfilt(b, a, svbe);
normfactor = np.max(np.fabs(svbe[:]));
if (cut_percentile is not None):
fx = np.fabs(svbe);
pv = np.percentile(fx, cut_percentile);
pvlow = np.percentile(fx, cut_percentile-1);
normfactor = pv;
ptile = (svbe > pv).astype(float);
pntile = (svbe < -pv).astype(float);
pboth = ptile+pntile;
einds = np.flatnonzero(pboth);
lastind = -2;
for j in range(len(einds)):
if(einds[j]==(lastind+1)):
svbe[einds[j]]=0;
else:
svbe[einds[j]]=pvlow;
if(normalize is not False):
svbe = np.true_divide(svbe, normfactor);
return svbe;
def getVisibleImpacts(self, force_recompute=False, include_cut_events = None, **kwargs):
feature_name = 'visible_impacts';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
svbe = self.getFeature('impact_envelope', **kwargs);
upsample_factor = kwargs.get('upsample_factor');
if(upsample_factor is None):
upsample_factor = 1;
u_sampling_rate = self.sampling_rate*upsample_factor;
peak_params = self._getDefaultPeakPickingTimeParams();
peak_params.update(kwargs); # if params given in arguments, those will override the defaults here.
v_events = Event.FromSignalPeaks(signal=svbe, sampling_rate=u_sampling_rate, **peak_params);
if(include_cut_events):
cut_events = self.getFeature('cut_events');
v_events = v_events + cut_events;
Event.Sort(v_events);
result = self.visualBeatsFromEvents(v_events);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getForwardVisibleImpacts(self, force_recompute=False, include_cut_events = None, **kwargs):
feature_name = 'forward_visual_beats';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
local_saliency = self.getFeature('forward_visual_impact_envelope', **kwargs);
upsample_factor = kwargs.get('upsample_factor');
if(upsample_factor is None):
upsample_factor = 1;
u_sampling_rate = self.sampling_rate*upsample_factor;
v_events = Event.FromSignalPeaks(signal=local_saliency, sampling_rate=u_sampling_rate, event_type= 'forward', **kwargs);
if(include_cut_events):
cut_events = self.getFeature('cut_events');
v_events = v_events + cut_events;
Event.Sort(v_events);
result = Event.SetDirections(v_events, direction=Event.DIRECTION_FORWARD);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getBackwardVisibleImpacts(self, force_recompute=False, include_cut_events = None, **kwargs):
feature_name = 'backward_visual_beats';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
local_saliency = self.getFeature('backward_visual_impact_envelope', **kwargs);
upsample_factor = kwargs.get('upsample_factor');
if(upsample_factor is None):
upsample_factor = 1;
u_sampling_rate = self.sampling_rate*upsample_factor;
v_events = Event.FromSignalPeaks(signal=local_saliency, sampling_rate=u_sampling_rate, **kwargs);
if(include_cut_events):
cut_events = self.getFeature('cut_events');
v_events = v_events + cut_events;
Event.Sort(v_events);
# result = self.visualBeatsFromEvents(v_events);
result = Event.SetDirections(v_events, direction=Event.DIRECTION_BACKWARD);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def findAccidentalDanceSequences(self, target_n_beats = 7, n_samples=25, delta_range = None):
if(delta_range is None):
delta_range = [0.02, 0.5];
deltas = np.linspace(delta_range[0], delta_range[1], num=n_samples, endpoint=True);
deltapick = delta_range[0];
sequences = [];
for i, delta in enumerate(deltas):
peak_vars = self._getDefaultPeakPickingTimeParams(delta=delta);
sequences = self.getVisualBeatSequences(peak_vars=peak_vars, print_summary=False);
# print("Delta {} has top sequence with {} beats".format(delta, len(sequences[0])));
if(len(sequences[0])<=target_n_beats):
deltapick = delta;
break;
print(("Selected delta value {}".format(deltapick)));
return sequences;
def getVisualBeatSequences(self,
search_tempo=None,
target_period=None,
search_window=0.75,
min_beat_limit=None,
max_beat_limit=None,
unary_weight=None,
binary_weight=None,
break_on_cuts=None,
peak_vars=None,
time_range=None,
n_return = None,
unsorted = False,
print_summary = True,
**kwargs):
"""
:param self:
:param search_tempo: optional tempo you would like to find visual beats at
:param target_period: optional target period you would like to use for finding beats. Ignored if search tempo is provided.
:param search_window: longest amount of time (seconds) allowed between beats before a segment is broken into multiple segments
:param min_beat_limit: only consider sequences with
:param unary_weight:
:param binary_weight:
:param break_on_cuts:
:param peak_vars:
:param kwargs:
:return:
"""
if (peak_vars is not None):
# impacts = self.getFeature('visible_impacts', force_recompute=True, **peak_vars);
impacts = self.getVisualBeats(force_recompute = True, **peak_vars);
else:
# impacts = self.getFeature('visible_impacts');
impacts = self.getVisualBeats();
if(time_range is not None):
impactseg = [];
for i in impacts:
if(i.start>time_range[0] and i.start < time_range[1]):
impactseg.append(i);
impacts = impactseg;
if (search_tempo is not None):
tempo = search_tempo;
beat_time = np.true_divide(60.0, tempo);
sequences = VisualBeat.PullOptimalPaths_Basic(impacts, target_period=beat_time, unary_weight=unary_weight,
binary_weight=binary_weight, break_on_cuts=break_on_cuts,
window_size=search_window);
elif(target_period is not None):
sequences = VisualBeat.PullOptimalPaths_Basic(impacts, target_period=target_period, unary_weight=unary_weight,
binary_weight=binary_weight, break_on_cuts=break_on_cuts,
window_size=search_window);
else:
sequences = VisualBeat.PullOptimalPaths_Autocor(impacts, unary_weight=unary_weight, binary_weight=binary_weight,
break_on_cuts=break_on_cuts, window_size=search_window);
r_sequences = [];
if(min_beat_limit is None):
min_beat_limit = 2;
if(max_beat_limit is None):
max_beat_limit = len(impacts)+1;
for S in sequences:
if ((len(S) > min_beat_limit) and (len(S) < max_beat_limit)):
r_sequences.append(S);
if(not unsorted):
r_sequences.sort(key=len, reverse=True);
if(n_return is not None):
r_sequences = r_sequences[:n_return];
if(print_summary):
print(("{} segments".format(len(r_sequences))));
for s in range(len(r_sequences)):
print(("Segment {} has {} beats".format(s, len(r_sequences[s]))));
return r_sequences;
def printVisualBeatSequences(self,
search_tempo=None,
target_period=None,
search_window=None,
min_beat_limit=None,
max_beat_limit=None,
unary_weight=None,
binary_weight=None,
break_on_cuts=None,
peak_vars=None,
n_return = None,
time_range=None, **kwargs):
"""
:param self:
:param target_period:
:param search_tempo:
:param search_window:
:param beat_limit:
:param unary_weight:
:param binary_weight:
:param break_on_cuts:
:param n_return:
:param peak_vars:
:param time_range:
:param kwargs:
:return:
"""
sorted = True;
sequence_args = dict(
search_tempo=search_tempo,
target_period=target_period,
search_window=search_window,
min_beat_limit=min_beat_limit,
max_beat_limit=max_beat_limit,
unary_weight=unary_weight,
binary_weight=binary_weight,
break_on_cuts=break_on_cuts,
peak_vars=peak_vars,
n_return=n_return,
time_range=time_range);
seqs = self.getVisualBeatSequences(**sequence_args)
print(("sequence arguments were:\n{}".format(sequence_args)));
print(("There were {} sequences total".format(len(seqs))));
nclips = 0;
rsegments = [];
for S in seqs:
if (len(S) > 1):
nclips = nclips + 1;
rsegments.append(S);
# rsegments.sort(key=len, reverse=True);
# if (n_return is not None):
# rsegments = rsegments[:n_return];
Event.PlotSignalAndEvents(self.getFeature('impact_envelope'), sampling_rate=self.sampling_rate*VB_UPSAMPLE_FACTOR, events=rsegments[0], time_range=time_range);
return rsegments;
def plotEvents(self, events, time_range = 'default', **kwargs):
time_range_use = time_range;
if(time_range.lower() == 'default'):
time_range_use = [0,0];
time_range_use[0] = events[0].start-1;
time_range_use[1] = events[-1].start+1;
signal = self.getFeature('local_rhythmic_saliency');
mplt = Event.PlotSignalAndEvents(signal, sampling_rate=self.sampling_rate * VB_UPSAMPLE_FACTOR, events=events, time_range=time_range_use, **kwargs);
plt.xlabel('Time (s)')
return mplt;
def plotCutEvents(self, **kwargs):
signal = self.getFeature('impact_envelope');
events = self.getFeature('cut_events', **kwargs);
Event.PlotSignalAndEvents(signal, sampling_rate=self.sampling_rate*VB_UPSAMPLE_FACTOR, events=events, **kwargs);
def plotVisibleImpacts(self, **kwargs):
signal = self.getFeature('impact_envelope');
events = self.getFeature('visible_impacts', **kwargs);
Event.PlotSignalAndEvents(signal, sampling_rate=self.sampling_rate*VB_UPSAMPLE_FACTOR, events=events, **kwargs);
plt.title('Impact Envelope & Visual Beats')
plt.xlabel('Time (s)')
plt.ylabel('Impact Strength')
def plotImpactEnvelope(self, **kwargs):
signal = self.getFeature('local_rhythmic_saliency');
# events = self.getFeature('visual_beats', **kwargs);
events = None;
Event.PlotSignalAndEvents(signal, sampling_rate=self.sampling_rate*VB_UPSAMPLE_FACTOR, events=events, **kwargs);
plt.title('Impact Envelope & Visual Beats')
plt.xlabel('Time (s)')
plt.ylabel('Impact Strength')
def plotVisualBeats(self, **kwargs):
signal = self.getFeature('local_rhythmic_saliency');
events = self.getFeature('visual_beats', **kwargs);
Event.PlotSignalAndEvents(signal, sampling_rate=self.sampling_rate*VB_UPSAMPLE_FACTOR, events=events, **kwargs);
plt.title('Impact Envelope & Visual Beats')
plt.xlabel('Time (s)')
plt.ylabel('Impact Strength')
def loadFlowFeatures(self):
self.load(features_to_load=['directogram_powers', 'directogram']);
FEATURE_FUNCS['local_rhythmic_saliency'] = getLocalRhythmicSaliency;
FEATURE_FUNCS['directogram_powers'] = getDirectogramPowers;
FEATURE_FUNCS['directogram'] = getDirectogram;
FEATURE_FUNCS['impact_envelope'] = getVisibleImpactEnvelope;
FEATURE_FUNCS['impact_envelope_powers'] = getVisibleImpactEnvelopePowers;
FEATURE_FUNCS['visible_impacts'] = getVisibleImpacts;
FEATURE_FUNCS['visual_beats'] = getVisualBeats;
# FEATURE_FUNCS['backward_visual_beats'] = getBackwardVisualBeats;
# FEATURE_FUNCS['forward_visual_beats'] = getForwardVisualBeats;
FEATURE_FUNCS['backward_visual_impact_envelope'] = getBackwardVisibleImpactEnvelope;
FEATURE_FUNCS['both_way_visual_impact_envelope'] = getBothWayVisibleImpactEnvelope;
FEATURE_FUNCS['forward_visual_impact_envelope'] = getForwardVisibleImpactEnvelope;
FEATURE_FUNCS['directional_flux'] = getDirectionalFlux;
FEATURE_FUNCS['visual_tempogram'] = getVisualTempogram;
FEATURE_FUNCS['cut_events']=getCutEvents;
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/VisBeatDefines.py
================================================
VB_DEBUG = True;
DEFAULT_TEMPOGRAM_WINDOW_SECONDS = 5;
AUDIO_DEFAULT_HOP_LENGTH = 512;
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/VisBeatExampleVideo.py
================================================
import os
from visbeat3.SourceLocationParser import ParseSourseLocation
class VisBeatExampleVideo(object):
def __init__(self, name, url, start_beat=None, end_beat = None, display_name = None, code=None, leadin=None, **kwargs):
self.name = name;
self.url = url;
self.start_beat = start_beat;
self.end_beat = end_beat;
self._display_name = display_name;
self._code = code;
self.leadin = leadin;
for k in kwargs:
setattr(self, k, kwargs[k]);
if(code is None):
sloc = ParseSourseLocation(self.url);
self._code = sloc.code;
#
@property
def code(self):
return self._getCode();
def _getCode(self):
return self._code;
@code.setter
def code(self, value):
self._setCode(value);
def _setCode(self, value):
self._code = value;
#
# def _ytEmbedCode(self):
#
@property
def display_name(self):
return self._getDisplayName();
def _getDisplayName(self):
if(self._display_name is None):
return self.name;
else:
return self._display_name;
#
def _ytWatchURL(self):
return 'https://www.youtube.com/watch?v={}'.format(self.code);
def _ytEmbedURL(self, autoplay=None):
s = 'https://www.youtube.com/embed/{}'.format(self.code);
if(autoplay):
s = s+'?autoplay=1';
return s;
def _ytThumbURL(self):
return 'https://ytimg.googleusercontent.com/vi/{}/default.jpg'.format(self.code);
def _fancyBoxCode(self, with_label = None):
""""""
s = HTMLCode();
if(with_label):
s.addLine("")
s.addLine('
'.format(watchurl=self._ytWatchURL(),
alt_name = self.display_name,
thumburl = self._ytThumbURL()))
if(with_label):
s.addLine('');
# s.addLine('{displayname}'.format(self.url, self.display_name));
s.addLine('{displayname}'.format(displayname=self.display_name));
s.addLine(' ')
s.addLine(' ');
return s.string;
from bs4 import BeautifulSoup
class HTMLCode(object):
def __init__(self, start_string=None):
self._code = "";
self._lines = [];
if(start_string is not None):
self.add(start_string)
self._soup = None;
#
@property
def string(self):
return self._getString();
def _getString(self):
return BeautifulSoup(self.code).prettify();
#
@property
def code(self):
return self._getCode();
def _getCode(self):
return self._code;
@code.setter
def code(self, value):
self._setCode(value);
def _setCode(self, value):
self._code = value;
#
#
@property
def soup(self):
return self._getSoup();
def _getSoup(self):
if(self._soup is None):
self._makeSoup();
return self._soup;
def _makeSoup(self):
self._soup = BeautifulSoup(self.code, 'html.parser');
# @soup.setter
# def soup(self, value):
# self._setSoup(value);
# def _setSoup(self, value):
# self._soup = value;
#
def add(self, s):
self.code = self.code+s;
def addLine(self, s):
self.code = self.code+'\n'+s;
def startTable(self, id=None, class_ = None, **kwargs):
targs = dict(table_width="80%", border=1, cellspacing=1, cellpadding=1)
targs.update(kwargs);
s = ''.format(**targs);
self.addLine(s);
self.addLine('');
def endTable(self):
self.addLine("");
self.addLine("
");
def startRow(self):
self.addLine('');
def endRow(self):
self.addLine(' ');
def addColumnLabel(self, label):
self.addLine('')
self.addLine(label);
self.addLine(' ');
def addRowLabel(self, label):
self.addLine('')
self.addLine(label);
self.addLine(' ');
def addRowCell(self, content):
self.addLine("");
self.addLine("")
self.addLine(content);
self.addLine("")
self.addLine(" ");
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/VisBeatImports.py
================================================
from .VisBeatDefines import *
from .AImports import *
from .AObject import AObject
import numpy as np
import scipy as sp
import os
import imageio
import matplotlib
try:
import matplotlib.pyplot as plt
except ImportError as e:
AWARN("matplotlib problem... if you are using conda try installing with 'conda install matplotlib'")
matplotlib.use('agg');
import matplotlib.pyplot as plt
import matplotlib.style as ms
import io
import base64
import math
from operator import truediv
import time
import shutil
from time import gmtime, strftime, localtime
import librosa;
from ._mediafiles import GetVBMarkPath
def local_time_string():
return strftime("%Y-%m-%d_%H:%M:%S", localtime());
def VBWARN(message):
print(message)
#
# def send_warnings_to_print_red(message, category, filename, lineno):
# print(colored('{} WARNING! file: {} Line:{}\n{}'.format(category, filename, lineno, message), 'red'))
# old_showwarning = warnings.showwarning
# warnings.showwarning = send_warnings_to_print_red;
VB_MACHINE_ID = None;
# if(VB_MACHINE_ID):
# matplotlib.use('PS');
ISNOTEBOOK = False;
if(runningInNotebook()):
ISNOTEBOOK = True;
import IPython;
import IPython.display
ms.use('seaborn-muted')
if(not runningInSpyder()):
get_ipython().magic('matplotlib inline')
from IPython.lib import kernel
connection_file_path = kernel.get_connection_file()
connection_file = os.path.basename(connection_file_path)
kernel_id = connection_file.split('-', 1)[1].split('.')[0]
# print("Kernel ID:\n{}".format(kernel_id));
from IPython.display import HTML
VBIPY = IPython;
#%matplotlib inline
# else:
# matplotlib.use('PS');
def vb_get_ipython():
return VBIPY;
def clipping_params(clip_bins=30, clip_fraction=0.95):
return dict(clip_bins=clip_bins, clip_fraction=clip_fraction);
def get_hist_clipped(signal, clip_bins=30, clip_fraction=0.95):
holdshape = signal.shape[:];
sigrav = signal.copy().ravel();
sigh, sigb = np.histogram(sigrav, bins=clip_bins);
maxbini=np.argmax(sigh);
totalmass = np.sum(sigh);
total_included = truediv(sigh[maxbini],totalmass);
prevbini = maxbini;
nextbini=maxbini;
bins_included = 1;
icounter=0;
while(total_included=0 and sigh[prevbini]==0) and (nextbini<(clip_bins) and sigh[nextbini]==0)):
prevbini=prevbini-1;
nextbini=nextbini+1;
else:
if((prevbini>=0 and sigh[prevbini]>0) or nextbini>=clip_bins):
prevbini=prevbini-1;
if((nextbini<(clip_bins) and sigh[nextbini]>0) or prevbini<0):
nextbini=nextbini+1;
included_segment=sigh[max(prevbini,0):min(nextbini+1, clip_bins)];
total_included = truediv(np.sum(included_segment), totalmass);
bins_included=len(included_segment);
icounter+=1;
clipsig = np.clip(a=sigrav, a_min=sigb[max(prevbini,0)], a_max=sigb[min(nextbini,clip_bins)]);
clipsig.shape=signal.shape
return clipsig;
def np_scale_to_range(data, value_range=None):
if(value_range is None):
value_range = [0.0,255.0];
d = data.copy().ravel();
currentmin=np.min(d);
currentmax=np.max(d);
currentscale = currentmax-currentmin;
if(currentscale==0):
VBWARN("CANNOT SCALE CONSTANT ARRAY TO RANGE")
return;
divscale = truediv(1.0,currentscale);
newrange=(value_range[1]-value_range[0]);
d = (d*divscale)*newrange;
newmin = (currentmin*divscale)*newrange;
d = d-newmin+value_range[0]
d.shape=data.shape;
return d
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/VisualBeat.py
================================================
from .Event import *
class VisualBeat(Event):
def VBOBJECT_TYPE(self):
return 'VisualBeat';
def __init__(self, start=None, type=None, weight=None, index=None, unrolled_start = None, direction=0):
Event.__init__(self, start=start, type=type, weight=weight, index=index, unrolled_start=unrolled_start, direction=direction);
def initializeBlank(self):
Event.initializeBlank(self);
self.flow_histogram = None;
self.local_autocor = None;
self.sampling_rate = None;
def __str__(self):
return "start:{}\ntype:{}\nweight:{}\nindex:{}\nunrolled_start:{}\nis_active:{}\n".format(self.start, self.type, self.weight, self.index, self.unrolled_start, self.is_active);
# def toGUIDict(self, is_active=1):
# return dict(start=self.start, index=self.index, is_active=None)
def toDictionary(self):
d=Event.toDictionary(self);
# d['start']=self.start;
# d['type']=self.type;
# d['weight']=self.weight;
# d['index']=self.index;
# d['unrolled_start']=self.unrolled_start;
d['flow_histogram']=self.flow_histogram;
d['local_autocor']=self.local_autocor;
d['sampling_rate']=self.sampling_rate;
return d;
def initFromDictionary(self, d):
Event.initFromDictionary(self, d);
self.start = d['start'];
self.type = d['type'];
self.weight = d['weight'];
self.index = d['index'];
self.unrolled_start = d['unrolled_start'];
self.flow_histogram = d.get('flow_histogram');
self.local_autocor = d.get('local_autocor');
self.sampling_rate = d.get('sampling_rate');
def clone(self, start=None):
if(start):
newv = VisualBeat(start = start, type=self.type, weight=self.weight, index=self.index);
else:
newv = VisualBeat(start = self.start, type=self.type, weight=self.weight, index=self.index);
newv.flow_histogram = self.flow_histogram.copy();
newv.local_autocor = self.local_autocor.copy();
newv.sampling_rate = self.sampling_rate;
return newv;
@staticmethod
def FromEvent(e):
if(isinstance(e,VisualBeat)):
return e.clone();
else:
return VisualBeat(start=e.start, type=e.type, weight=e.weight, index=e.index);
@staticmethod
def time_window_func(max_separation, break_on_cuts=None):
def window_func(a, b):
if(break_on_cuts and (a.type=='cut' or b.type=='cut')):
return False;
if(np.fabs(a.start-b.start)3.75):
score=-1;
return binary_weight*score;
return objective_func;
@staticmethod
def angle_binary_objective(binary_weight=None, absolute=None):
if (binary_weight is None):
binary_weight = 1.0;
# if (angle_weight is None):
# angle_weight = 0.0;
def objective_func(a, b):
# T = np.fabs(a.start - b.start);
# tempo_score = -np.power(np.log(truediv(T, target_period)), 2.0) * binary_weight;
# should mask out unary contribution from orthogonal angles
if(absolute):
return binary_weight * (np.fabs(np.dot(a.flow_histogram, b.flow_histogram)) - 0.70710678118); # cos 45 degrees
else:
return binary_weight * (np.dot(a.flow_histogram, b.flow_histogram)); # cos 45 degrees
return objective_func;
@staticmethod
def Double(events, type=None):
doubled = [];
for e in range(1, len(events)):
halfstart = 0.5 * (events[e].start + events[e - 1].start);
newhevent = events[e].clone(start=halfstart);
if(type is not None):
newhevent.type = type;
doubled.append(newhevent);
doubled.append(events[e]);
return doubled;
@staticmethod
def weight_unary_objective(unary_weight=None):
if(unary_weight is None):
unary_weight = 1.0;
def getweight_func(b):
return unary_weight*b.weight;
return getweight_func;
@staticmethod
def PullOptimalPaths_Basic(vis_beats, target_period, unary_weight=None, binary_weight=None, window_size=None, break_on_cuts = None):
if(window_size is None):
window_size = DEFAULT_WINDOW_FACTOR*target_period;
binary_objective = VisualBeat.tempo_binary_objective(target_period=target_period, binary_weight = binary_weight);
unary_objective = VisualBeat.weight_unary_objective(unary_weight=unary_weight);
window_function = VisualBeat.time_window_func(max_separation = window_size, break_on_cuts=break_on_cuts);
return VisualBeat.DynamicProgramOptimalPaths(vis_beats=vis_beats,
unary_objective_func=unary_objective,
binary_objective_func=binary_objective,
window_func=window_function);
@staticmethod
def PullOptimalPaths(vis_beats, unary_fn=None, binary_fn=None, window_fn=None, target_period=None, unary_weight=None, binary_weight=None, window_size=None,
break_on_cuts=None):
if (window_size is None):
window_size = DEFAULT_WINDOW_FACTOR * target_period;
if(binary_fn == 'autocor'):
binary_objective = VisualBeat.autocor_binary_objective(binary_weight=binary_weight);
elif(binary_fn == 'angle'):
binary_objective = VisualBeat.angle_binary_objective(binary_weight=binary_weight);
else:
binary_objective = VisualBeat.tempo_binary_objective(target_period=target_period, binary_weight=binary_weight);
unary_objective = VisualBeat.weight_unary_objective(unary_weight=unary_weight);
window_function = VisualBeat.time_window_func(max_separation=window_size, break_on_cuts=break_on_cuts);
return VisualBeat.DynamicProgramOptimalPaths(vis_beats=vis_beats,
unary_objective_func=unary_objective,
binary_objective_func=binary_objective,
window_func=window_function);
@staticmethod
def PullOptimalPaths_Autocor(vis_beats, unary_weight=None, binary_weight=None, window_size=None,
break_on_cuts=None, **kwargs):
if (window_size is None):
# assert(False), 'no window size provided'
# window_size = DEFAULT_WINDOW_FACTOR * target_period;
window_size = 200;
AWARN('NO WINDOWSIZE PROVIDED! PullOptimalPaths_Autocor');
binary_objective = VisualBeat.autocor_binary_objective(binary_weight=binary_weight);
unary_objective = VisualBeat.weight_unary_objective(unary_weight=unary_weight);
window_function = VisualBeat.time_window_func(max_separation=window_size, break_on_cuts=break_on_cuts);
return VisualBeat.DynamicProgramOptimalPaths(vis_beats=vis_beats,
unary_objective_func=unary_objective,
binary_objective_func=binary_objective,
window_func=window_function);
@staticmethod
def DynamicProgramOptimalPaths(vis_beats, unary_objective_func, binary_objective_func, window_func):
class Node(object):
def __init__(self, object, prev_node=None):
self.object = object;
self.prev_node = prev_node;
self.cum_score=None;
nodes = [];
beats = Event.GetSorted(vis_beats);
Event.ApplyIndices(beats);
for b in beats:
nodes.append(Node(object=b));
nodes[0].prev_node = None;
nodes[0].cum_score = unary_objective_func(nodes[0].object);
current_segment = [];
segments = [];
for n in range(1,len(nodes)):
current_node = nodes[n];
current_segment.append(current_node);
options = [];
j = n-1;
while(j>=0 and window_func(current_node.object,nodes[j].object)):
options.append(nodes[j]);
j = j-1;
if(len(options)==0):
current_node.prev_node=None;
current_node.cum_score=unary_objective_func(current_node.object);
segments.append(current_segment);
current_segment = [];
else:
best_choice = options[0];
best_score = options[0].cum_score+binary_objective_func(current_node.object, best_choice.object);
for o in range(1,len(options)):
score = options[o].cum_score+binary_objective_func(current_node.object, options[o].object);
if(score>best_score):
best_choice=options[o];
best_score=score;
current_node.prev_node = best_choice;
current_node.cum_score = best_score+unary_objective_func(current_node.object);
if(len(current_segment)>0):
segments.append(current_segment);
sequences = [];
for S in segments:
seq = [];
max_node = S[0];
max_score = max_node.cum_score;
for n in range(len(S)):
if(S[n].cum_score>max_score):
max_node = S[n];
max_score = max_node.cum_score;
trace_node = max_node;
while(trace_node.prev_node is not None):
seq.append(trace_node.object);
trace_node=trace_node.prev_node;
seq.reverse();
sequences.append(seq);
return sequences;
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/Warp.py
================================================
from .VisBeatImports import *
from .EventList import *
import math
DEFAULT_LEAD_TIME = 0;
DEFAULT_TAIL_TIME = 0;
class Warp(AObject):
"""Warp (class): defines how one time signal should be warped to another. Given primarily as source/target events to be matched.
Attributes:
source_events: source events
target_events: target events
"""
def VBOBJECT_TYPE(self):
return 'Warp';
def __init__(self, path=None):
AObject.__init__(self, path=path);
if(path):
self.loadFile();
@staticmethod
def FromEvents(source_events, target_events):
w = Warp();
sevents = source_events;
if(isinstance(source_events, EventList)):
sevents = source_events.events;
tevents = target_events;
if(isinstance(target_events, EventList)):
tevents = target_events.events;
w.source_events=sevents;
w.target_events=tevents;
# w.repeatShorterEvents();
return w
@staticmethod
def FromEventLists(source_eventlist, target_eventlist):
w = Warp();
w.source_events = source_eventlist.events;
w.target_events = target_eventlist.events;
# w.repeatShorterEvents();
return w
def initializeBlank(self):
AObject.initializeBlank(self);
self.source_events = [];
self.target_events = [];
# self.a_info['WarpType'] = 'Linear';
self.warp_func = None; # Warp.LinearInterp;
# self.warp_func_st = None;
# self.warp_func_ts = None;
def getTargetStart(self, lead=None):
target_start = self.target_events[0].getStartTime();
if (lead is None):
lead = min(target_start, DEFAULT_LEAD_TIME);
return target_start - lead;
def getTargetEnd(self, lead=None):
lastind = min(len(self.source_events), len(self.target_events)) - 1;
return self.target_events[lastind].getStartTime() + DEFAULT_TAIL_TIME;
def getSourceStart(self):
source_start = self.source_events[0].getUnrolledStartTime();
return source_start;
def getSourceEnd(self):
lastind = min(len(self.source_events), len(self.target_events)) - 1;
return self.source_events[lastind].getUnrolledStartTime();
def getWarpedSourceStart(self, lead=None):
source_start = self.source_events[0].getUnrolledStartTime();
if (lead is None):
lead = min(source_start, DEFAULT_LEAD_TIME);
return self.warpSourceTime(source_start - lead);
def getWarpedSourceEnd(self, tail=None):
last_event = min(len(self.source_events), len(self.target_events)) - 1;
source_end = self.source_events[last_event].getUnrolledStartTime();
if (tail is None):
tail = DEFAULT_TAIL_TIME;
source_end = source_end + tail;
return self.warpSourceTime(source_end);
def setWarpFunc(self, warp_type, **kwargs):
if (warp_type == 'square'):
self.warp_func = [Warp.SquareInterp, Warp.SquareInterp];
elif (warp_type == 'linear'):
self.warp_func = [Warp.LinearInterp, Warp.LinearInterp];
elif (warp_type == 'cubic'):
self.warp_func = [Warp.CubicInterp, Warp.CubicInterp];
elif (warp_type == 'quad'):
self.warp_func = [
Warp.GetEventWarpFunc(self.source_events, self.target_events, Warp.WFunc_Quadratic(), **kwargs),
Warp.GetEventWarpFunc(self.target_events, self.source_events, Warp.WFunc_Quadratic(), **kwargs)];
elif (warp_type == 'mouth'):
self.warp_func = [
Warp.GetEventWarpFunc(self.source_events, self.target_events, Warp.WFunc_Mouth(**kwargs), **kwargs),
Warp.GetEventWarpFunc(self.target_events, self.source_events, Warp.WFunc_Mouth(**kwargs), **kwargs)];
elif (warp_type == 'weight'):
self.warp_func = [
Warp.GetEventWarpFunc(self.source_events, self.target_events,
Warp.WFunc_Weight(use_to_weights=None, **kwargs), **kwargs),
Warp.GetEventWarpFunc(self.target_events, self.source_events,
Warp.WFunc_Weight(use_to_weights=True, **kwargs), **kwargs)];
elif (warp_type == 'half_accel'):
self.warp_func = [
Warp.GetEventWarpFunc(self.source_events, self.target_events, Warp.WFunc_P(p=0.5),
**kwargs),
Warp.GetEventWarpFunc(self.target_events, self.source_events, Warp.WFunc_P(p=0.5),
**kwargs)];
elif (warp_type == 'p'):
p = kwargs.get('p');
self.warp_func = [
Warp.GetEventWarpFunc(self.source_events, self.target_events, Warp.WFunc_P(p=p),
**kwargs),
Warp.GetEventWarpFunc(self.target_events, self.source_events, Warp.WFunc_P(p=p),
**kwargs)];
elif (warp_type == 'target_time_source_fraction'):
self.warp_func = [
Warp.GetEventWarpFunc(self.source_events, self.target_events,
Warp.WFunc_targettime_sourcefraction(**kwargs),
**kwargs),
Warp.GetEventWarpFunc(self.target_events, self.source_events,
Warp.WFunc_targettime_sourcefraction(**kwargs),
**kwargs)];
elif (warp_type == 'target_source_fractions'):
self.warp_func = [
Warp.GetEventWarpFunc(self.source_events, self.target_events,
Warp.WFunc_target_source_fractions(**kwargs),
**kwargs),
Warp.GetEventWarpFunc(self.target_events, self.source_events,
Warp.WFunc_target_source_fractions(**kwargs),
**kwargs)];
elif (warp_type is not None):
self.warp_func = [warp_type, warp_type];
self.setInfo('WarpType', warp_type);
return;
def warpSourceTime(self, t):
return self.warp_func[0](t, a_events=self.source_events, b_events=self.target_events);
def warpSourceTimes(self, t):
tw = t.copy();
for a in range(len(t)):
tw[a] = self.warpSourceTime(t[a]);
return tw;
def warpTargetTime(self, t):
return self.warp_func[1](t, a_events=self.target_events, b_events=self.source_events);
def warpTargetTimes(self, t):
tw = t.copy();
for a in range(len(t)):
tw[a] = self.warpTargetTime(t[a]);
return tw;
def plot(self, xlim=None, sampling_rate=None, new_figure=None, render_control_points=True, render_labels=True,
time_range=None, full_source_range=None, **kwargs):
if (sampling_rate is None):
sampling_rate = 30;
source_duration = self.getSourceEnd() - self.getSourceStart();
old_frame_time = truediv(1.0, sampling_rate);
target_start = self.getTargetStart();
target_end = self.getTargetEnd();
# if(xlim is not None):
# target_start=xlim[0]
target_duration = target_end - target_start;
new_n_samples = target_duration * sampling_rate;
target_start_times = np.linspace(target_start, target_end, num=new_n_samples, endpoint=False);
unwarped_target_times = [];
for st in target_start_times:
unwarped_target_times.append(self.warpTargetTime(st));
if (new_figure):
fig = plt.figure();
unwarped_target_times = np.array(unwarped_target_times);
# unwarped_target_times = unwarped_target_times-unwarped_target_times[0]+target_start;
plt.plot(target_start_times, unwarped_target_times, '-');
if (render_control_points):
lastind = min(len(self.source_events), len(self.target_events)) - 1;
targeteventtimes = Event.ToStartTimes(self.target_events[:lastind]);
sourceeventtimes = Event.ToStartTimes(self.source_events[:lastind]);
plt.plot(targeteventtimes, sourceeventtimes, 'o', label='Control Points');
if (xlim is not None):
xrng = [xlim[0] + target_start, xlim[1] + target_start];
ylim = [self.warpTargetTime(xrng[0]), self.warpTargetTime(xrng[1])];
plt.xlim(xrng);
plt.ylim(ylim);
if (time_range is not None):
plt.xlim(time_range);
if (render_labels):
plt.ylabel('Source Time');
plt.xlabel('Target Time');
plt.title('Warp Curve')
if (new_figure is not None):
return fig;
def plotImage(self, xlim=None, sampling_rate=None):
if (sampling_rate is None):
sampling_rate = 30;
target_start = self.getTargetStart();
target_end = self.getTargetEnd() + 10;
target_duration = target_end - target_start;
new_n_samples = target_duration * sampling_rate;
target_start_times = np.linspace(target_start, target_end, num=new_n_samples, endpoint=True);
unwarped_target_times = [];
for st in target_start_times:
unwarped_target_times.append(self.warpTargetTime(st));
unwarped_target_times = np.array(unwarped_target_times);
pim = Image.PlotImage(signal=unwarped_target_times, show_axis=True, xvals=target_start_times,
sampling_rate=sampling_rate, events=self.target_events, xlime=[0, 100], ylims=[0, 110]);
return pim;
def repeatShorterEvents(self, endpoints=False):
n_events = max(len(self.source_events), len(self.target_events));
self.source_events = Event.RepeatToLength(self.source_events, n=n_events, endpoints=endpoints);
self.target_events = Event.RepeatToLength(self.target_events, n=n_events, endpoints=endpoints);
@staticmethod
def FromEvents(source_events, target_events):
w = Warp();
w.source_events = source_events;
w.target_events = target_events;
# w.repeatShorterEvents();
return w
@staticmethod
def LinearInterp(t, a_events, b_events):
n_events = min(len(a_events), len(b_events));
next_a_event_index = n_events;
for s in range(n_events):
if (t < a_events[s].start):
next_a_event_index = s;
break;
prev_a_event_time = 0;
prev_b_event_time = 0;
if (next_a_event_index > 0):
prev_a_event_time = a_events[next_a_event_index - 1].start;
prev_b_event_time = b_events[next_a_event_index - 1].start;
next_a_event_time = a_events[n_events - 1].start;
next_b_event_time = b_events[n_events - 1].start;
if (next_a_event_index < n_events):
next_a_event_time = a_events[next_a_event_index].start;
next_b_event_time = b_events[next_a_event_index].start;
a_event_gap = next_a_event_time - prev_a_event_time;
# b_event_gap = next_b_event_time - prev_b_event_time;
t_progress = t - prev_a_event_time;
# take care of past-the-end case, by simply letting time proceed normally past the last event
if (a_event_gap == 0):
return prev_b_event_time + t_progress;
next_weight = t_progress / (1.0 * a_event_gap);
return (next_weight * next_b_event_time) + ((1.0 - next_weight) * prev_b_event_time);
# additional_points = [];
# for i in range(-10, 11):
# additional_points.append([0.1 * i, 0.51])
@staticmethod
def plotWarpMethodTest(warp_type, additional_points=None, **kwargs):
sev = [];
tev = [];
pts = [[1.0, 1.0],
[-1.0, 1.0],
[0.5, 1.0],
[1.0, 0.5],
[-1.0, 0.5],
[0.5, 0.5],
[1.0, 0.3],
[-1.0, 0.3],
[0.5, 0.3]];
if (additional_points is not None):
pts = pts + additional_points;
currentt = [0.0, 0.0];
sev.append(Event(start=0.0));
tev.append(Event(start=0.0));
for p in pts:
currentt[0] = currentt[0] + p[0];
currentt[1] = currentt[1] + p[1];
sev.append(Event(start=currentt[0]));
tev.append(Event(start=currentt[1]));
# warp_type = 'target_time_source_fraction';
# other_params['acceleration_target_time'] = 0.5;
# other_params['acceleration_source_fraction'] = 0.75;
warpf = Warp.FromEvents(sev, tev);
warpf.setWarpFunc(warp_type, **kwargs);
warpf.plot()
@staticmethod
def SquareInterp(t, a_events, b_events):
n_events = min(len(a_events), len(b_events));
next_a_event_index = n_events;
for s in range(n_events):
if (t < a_events[s].start):
next_a_event_index = s;
break;
prev_a_event_time = 0;
prev_b_event_time = 0;
if (next_a_event_index > 0):
prev_a_event_time = a_events[next_a_event_index - 1].start;
prev_b_event_time = b_events[next_a_event_index - 1].start;
next_a_event_time = a_events[n_events - 1].start;
next_b_event_time = b_events[n_events - 1].start;
if (next_a_event_index < n_events):
next_a_event_time = a_events[next_a_event_index].start;
next_b_event_time = b_events[next_a_event_index].start;
a_event_gap = next_a_event_time - prev_a_event_time;
# b_event_gap = next_b_event_time - prev_b_event_time;
t_progress = t - prev_a_event_time;
# take care of past-the-end case, by simply letting time proceed normally past the last event
if (a_event_gap == 0):
return prev_b_event_time + t_progress;
progress_frac = t_progress / (1.0 * a_event_gap);
next_weight = math.pow(progress_frac, 2);
# accel = 3;
# bweight = math.pow(progress_frac,accel);
# aweight = math.pow(1.0-progress_frac,accel);
# sumweight = aweight+bweight;
# next_weight=bweight/sumweight;
return (next_weight * next_b_event_time) + ((1.0 - next_weight) * prev_b_event_time);
@staticmethod
def GetEventWarpFunc(from_events, to_events, f, lead_time=None, **kwargs):
start_cap_time = min(from_events[0].start, to_events[0].start);
if (lead_time is not None):
start_cap_time = min(start_cap_time, lead_time);
n_events = min(len(from_events), len(to_events));
# f_events = Event.GetUnrolledList(from_events[:n_events], assert_on_folds=True);
f_events = Event.GetUnrolledList(from_events[:n_events]);
t_events = Event.GetUnrolledList(to_events[:n_events]);
def rfunc(t, **kwargs):
next_f_event_index = n_events - 1;
for e in range(n_events):
if (t < f_events[e].unrolled_start):
next_f_event_index = e;
break;
if (next_f_event_index == 0):
from_cap_event = Event(start=f_events[0].start - start_cap_time, weight=0, type='startcap');
to_cap_event = Event(start=t_events[0].start - start_cap_time, weight=0, type='startcap')
return f(t, f_neighbors=[from_cap_event, f_events[0]], t_neighbors=[to_cap_event, t_events[0]]);
else:
return f(t,
f_neighbors=[f_events[next_f_event_index - 1], f_events[next_f_event_index]],
t_neighbors=[t_events[next_f_event_index - 1], t_events[next_f_event_index]]);
return rfunc;
@staticmethod
def WFunc_Quadratic():
def rfunc(t, f_neighbors, t_neighbors, **kwargs):
from_times = Event.ToStartTimes(f_neighbors);
to_times = Event.ToStartTimes(t_neighbors);
from_event_gap = from_times[1] - from_times[0];
t_progress = t - from_times[0];
# avoid divide by 0
if (from_event_gap == 0):
AWARN("Event Gap was 0! Check Warp.py")
return prev_to_event_time + t_progress;
progress_frac = truediv(t_progress, from_event_gap);
next_weight = math.pow(progress_frac, 2);
return (next_weight * to_times[1]) + ((1.0 - next_weight) * to_times[0]);
return rfunc;
@staticmethod
def WFunc_Weight(use_to_weights=None, **kwargs):
print("USING WEIGHT-BASED WARP");
def rfunc(t, f_neighbors, t_neighbors):
from_times = Event.ToStartTimes(f_neighbors);
to_times = Event.ToStartTimes(t_neighbors);
from_event_gap = from_times[1] - from_times[0];
to_event_gap = to_times[1] - to_times[0];
t_progress = t - from_times[0];
# avoid divide by 0
if (from_event_gap == 0):
AWARN("Event Gap was 0! Check Warp.py")
return prev_to_event_time + t_progress;
progress_frac = truediv(t_progress, from_event_gap);
if (use_to_weights):
weight = t_neighbors[1].weight;
else:
weight = f_neighbors[1].weight;
p = 1.0 - weight;
# a = truediv(1.0, (1.0+p*p-2*p)); # if a=b
a = 1.0 - np.power((1.0 - p), 2.0); # b=1
if (progress_frac < p):
next_weight = a * progress_frac;
else:
next_weight = (a * progress_frac) + np.power((progress_frac - p), 2.0);
return (next_weight * to_times[1]) + ((1.0 - next_weight) * to_times[0]);
return rfunc;
@staticmethod
def WFunc_P(p=None, **kwargs):
if (p is None):
p = 0.5;
print(("USING P WARP with P={}".format(p)));
def rfunc(t, f_neighbors, t_neighbors):
from_times = Event.ToStartTimes(f_neighbors);
to_times = Event.ToStartTimes(t_neighbors);
from_event_gap = from_times[1] - from_times[0];
to_event_gap = to_times[1] - to_times[0];
t_progress = t - from_times[0];
# avoid divide by 0
if (from_event_gap == 0):
AWARN("Event Gap was 0! Check Warp.py")
return prev_to_event_time + t_progress;
progress_frac = truediv(t_progress, from_event_gap);
# a = truediv(1.0, (1.0+p*p-2*p)); # if a=b
a = 1.0 - np.power((1.0 - p), 2.0); # b=1
if (progress_frac < p):
next_weight = a * progress_frac;
else:
next_weight = (a * progress_frac) + np.power((progress_frac - p), 2.0);
return (next_weight * to_times[1]) + ((1.0 - next_weight) * to_times[0]);
return rfunc;
@staticmethod
def WFunc_Mouth(p_acceleration_time=0.1, **kwargs):
def rfunc(t, f_neighbors, t_neighbors):
from_times = Event.ToStartTimes(f_neighbors);
to_times = Event.ToStartTimes(t_neighbors);
from_event_gap = from_times[1] - from_times[0];
to_event_gap = to_times[1] - to_times[0];
t_progress = t - from_times[0];
# avoid divide by 0
if (from_event_gap == 0):
AWARN("Event Gap was 0! Check Warp.py")
return prev_to_event_time + t_progress;
progress_frac = truediv(t_progress, from_event_gap);
if (f_neighbors[1].type == 'mouth_open' or t_neighbors[1].type == 'mouth_open'):
p = 1.0 - truediv(p_acceleration_time, to_event_gap);
if (p < 0):
next_weight = math.pow(progress_frac, 2);
else:
# a = truediv(1.0, (1.0+p*p-2*p)); # if a=b
a = 1.0 - np.power((1.0 - p), 2.0); # b=1
if (progress_frac < p):
next_weight = a * progress_frac;
else:
next_weight = (a * progress_frac) + np.power((progress_frac - p), 2.0);
else:
next_weight = progress_frac;
return (next_weight * to_times[1]) + ((1.0 - next_weight) * to_times[0]);
return rfunc;
@staticmethod
def WFunc_targettime_sourcefraction(acceleration_target_time=0.1, acceleration_source_fraction=0.8, **kwargs):
"""
This assumes that you are mapping from the target to the source, as is the most common use case.
:param acceleration_target_time: amount of from time to spend accelerating
:param acceleration_source_fraction: fraction of source to accelerate through
:return:
"""
lin_source_fraction = 1.0 - acceleration_source_fraction;
def rfunc(t, f_neighbors, t_neighbors):
from_times = Event.ToStartTimes(f_neighbors);
to_times = Event.ToStartTimes(t_neighbors);
from_event_gap = from_times[1] - from_times[0];
to_event_gap = to_times[1] - to_times[0];
t_progress = t - from_times[0];
# avoid divide by 0
if (from_event_gap == 0):
AWARN("Event Gap was 0! Check Warp.py")
return to_times[
0] + t_progress; # is this right? doesnt seem to come up, not sure what behavior should be looking back on it...
progress_frac = truediv(t_progress, from_event_gap);
time_left = from_event_gap - t_progress;
p = 1.0 - truediv(acceleration_target_time, from_event_gap);
p2 = p * p;
q = 1.0 - acceleration_source_fraction;
if (acceleration_target_time >= from_event_gap or q > (p2)):
next_weight = math.pow(progress_frac, 2);
elif (time_left >= acceleration_target_time):
lin_t_progress_frac = truediv(t_progress, from_event_gap - acceleration_target_time);
next_weight = lin_t_progress_frac * lin_source_fraction;
else:
pdnom = (p2 - 2.0 * p + 1.0)
# I just used matlab to symbolic inverse matrix of equations, then simplified by hand
a = ((1.0 - q) / pdnom) + (q / (p * (p - 1)));
b = ((2 * p * (q - 1)) / pdnom) - ((q * (p + 1)) / (p2 - p))
# c = ((p2 - (q * (2 * p - 1))) / pdnom) + (q / (p - 1));
c = 1 - a - b;
next_weight = a * progress_frac * progress_frac + b * progress_frac + c;
return (next_weight * to_times[1]) + ((1.0 - next_weight) * to_times[0]);
return rfunc;
@staticmethod
def WFunc_target_source_fractions(acceleration_target_fraction=0.8, acceleration_source_fraction=0.9, **kwargs):
"""
This assumes that you are mapping from the target to the source, as is the most common use case.
:param acceleration_target_fraction: fraction of target to spend accelerating
:param acceleration_source_fraction: fraction of source to accelerate through
:return:
"""
lin_source_fraction = 1.0 - acceleration_source_fraction;
def rfunc(t, f_neighbors, t_neighbors):
print(acceleration_target_fraction)
print(acceleration_source_fraction)
from_times = Event.ToStartTimes(f_neighbors);
to_times = Event.ToStartTimes(t_neighbors);
from_event_gap = from_times[1] - from_times[0];
to_event_gap = to_times[1] - to_times[0];
t_progress = t - from_times[0];
# avoid divide by 0
if (from_event_gap == 0):
AWARN("Event Gap was 0! Check Warp.py")
return to_times[
0] + t_progress; # is this right? doesnt seem to come up, not sure what behavior should be looking back on it...
progress_frac = truediv(t_progress, from_event_gap);
time_left = from_event_gap - t_progress;
p = 1.0 - acceleration_target_fraction;
p2 = p * p;
q = 1.0 - acceleration_source_fraction;
if (acceleration_target_fraction >= 1 or q > (p2)):
next_weight = math.pow(progress_frac, 2);
elif (progress_frac <= p):
lin_t_progress_frac = truediv(t_progress, p * from_event_gap);
next_weight = lin_t_progress_frac * lin_source_fraction;
else:
pdnom = (p2 - 2.0 * p + 1.0)
# I just used matlab to symbolic inverse matrix of equations, then simplified by hand
a = ((1.0 - q) / pdnom) + (q / (p * (p - 1)));
b = ((2 * p * (q - 1)) / pdnom) - ((q * (p + 1)) / (p2 - p))
# c = ((p2 - (q * (2 * p - 1))) / pdnom) + (q / (p - 1));
c = 1 - a - b;
next_weight = a * progress_frac * progress_frac + b * progress_frac + c;
return (next_weight * to_times[1]) + ((1.0 - next_weight) * to_times[0]);
return rfunc;
@staticmethod
def WFunc_AB(const_factor, quad_factor, **kwargs):
def rfunc(t, f_neighbors, t_neighbors):
from_times = Event.ToStartTimes(f_neighbors);
to_times = Event.ToStartTimes(t_neighbors);
from_event_gap = from_times[1] - from_times[0];
t_progress = t - from_times[0];
# avoid divide by 0
if (from_event_gap == 0):
AWARN("Event Gap was 0! Check Warp.py")
return prev_to_event_time + t_progress;
progress_frac = truediv(t_progress, from_event_gap);
next_weight = math.pow(progress_frac, 2);
return (next_weight * to_times[1]) + ((1.0 - next_weight) * to_times[0]);
@staticmethod
def ABWarp():
next_from_event_time = from_events[n_events - 1].start;
next_to_event_time = to_events[n_events - 1].start;
if (next_from_event_index < n_events):
next_from_event_time = from_events[next_from_event_index].start;
next_to_event_time = to_events[next_to_event_index].start;
from_event_gap = next_from_event_time - prev_from_event_time;
# b_event_gap = next_b_event_time - prev_b_event_time;
t_progress = t - prev_from_event_time;
# take care of past-the-end case, by simply letting time proceed normally past the last event
if (from_event_gap == 0):
return prev_to_event_time + t_progress;
progress_frac = t_progress / (1.0 * from_event_gap);
next_weight = math.pow(progress_frac, 2);
return (next_weight * next_to_event_time) + ((1.0 - next_weight) * prev_to_event_time);
@staticmethod
def CubicInterp(t, a_events, b_events):
# def CubicInterp(a_events, b_events, t):
f = Warp.CubicInterpFunc(a_events=a_events, b_events=b_events);
return f(t);
@staticmethod
def LinearInterpFunc(a_events, b_events):
if (a_events[0].start > 0):
ae = np.concatenate((np.asarray([0]), Event.ToStartTimes(a_events)));
else:
ae = Event.ToStartTimes(a_events);
be = Event.ToStartTimes(b_events);
if (len(be) > len(ae)):
be = be[:len(ae)];
elif (len(ae) > len(be)):
ae = ae[:len(be)];
return sp.interpolate.interp1d(ae, be, 'linear', bounds_error=False, fill_value='extrapolate');
@staticmethod
def CubicInterpFunc(a_events, b_events):
event_times = Event.ToStartTimes(a_events);
wevent_times = Event.ToStartTimes(b_events);
minlen = min(len(event_times), len(wevent_times));
event_times = event_times[:minlen];
wevent_times = wevent_times[:minlen];
splinecap = np.arange(0, 1, 0.25);
spline_times = np.concatenate((splinecap + event_times[0] - 1, event_times));
wspline_times = np.concatenate((splinecap + wevent_times[0] - 1, wevent_times));
spline_times = np.append(spline_times, splinecap + 0.5 + spline_times[-1]);
wspline_times = np.append(wspline_times, splinecap + 0.5 + wspline_times[-1]);
# get spline to translate initial times to warped times
splineX = sp.interpolate.interp1d(spline_times, wspline_times, kind='cubic', bounds_error=False,
fill_value='extrapolate');
return splineX;
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/__init__.py
================================================
from .VisBeatImports import *
from .Video import *
from .VideoClip import *
from .VideoSource import *
from .VisBeatExampleVideo import VisBeatExampleVideo
import re
import os;
import shutil
from .SourceLocationParser import ParseSourseLocation
VISBEAT_ASSETS_DIR = './VisBeatAssets/';
from . import fileui
fileui.INITIAL_DIR = VISBEAT_ASSETS_DIR;
def SetAssetsDir(assets_dir):
global VISBEAT_ASSETS_DIR;
VISBEAT_ASSETS_DIR = assets_dir;
make_sure_dir_exists(assets_dir);
AINFORM("VISBEAT_ASSETS_DIR set to {}".format(VISBEAT_ASSETS_DIR));
make_sure_dir_exists(GetVideoSourcesDir());
temp_dir = os.path.join(VISBEAT_ASSETS_DIR, 'TEMP_FILES'+os.sep);
make_sure_dir_exists(temp_dir);
Video.VIDEO_TEMP_DIR = temp_dir;
fileui.INITIAL_DIR = VISBEAT_ASSETS_DIR;
def GetAssetsDir():
return VISBEAT_ASSETS_DIR;
def GetVideoSourcesDir():
video_sources_dir = os.path.join(GetAssetsDir(), 'VideoSources'+os.sep);
make_sure_dir_exists(video_sources_dir);
return video_sources_dir;
def PullVideo(name=None, source_location=None, max_height=240, **kwargs):
if(isinstance(name, VisBeatExampleVideo)):
assert(source_location is None), 'Provided VisBeatExampleVideo and source location? What are you trying to do?';
source_location = name.url;
vname = name.name;
elif(name is None):
assert(source_location is not None), "Must provide an argument to pullvideo";
sloc = ParseSourseLocation(source_location);
vname =sloc.code;
else:
vname = name;
vs = GetVideoSource(vname);
if(vs and vs.source_location==source_location):
v = vs.getVersion(max_height=max_height);
v.load(features_to_load = 'all');
return v;
print("destination:", GetVideoSourcesDir(), "name:", vname, "source_location:", source_location)
vs = VideoSource.NewVideoSource(destination=GetVideoSourcesDir(), name=vname, source_location=source_location, max_height=max_height, **kwargs);
v = vs.getVersion(max_height=max_height);
return v;
def ClipVideo(video, time_range, max_height=240):
video_fullres = video.source.getVersion();
vclip = video_fullres.VideoClip(start=time_range[0], end=time_range[1]);
vcdir = video.source.getDirForVersion(version_label='{}_{}'.format(str(time_range[0]), str(time_range[1])), version_group='Clips');
make_sure_dir_exists(vcdir);
vcname = video.getName() + 'clip_{}_{}'.format(str(time_range[0]), str(time_range[1]));
vcpath = os.path.join(vcdir, vcname+'.mp4');
vclip.write(output_path=vcpath);
vs = VideoSource.NewVideoSource(destination=GetVideoSourcesDir(), name=vcname, source_location=vcpath);
return vs.getVersion(max_height=max_height);
def GetVideoSource(name):
vname = name;
if (isinstance(name, VisBeatExampleVideo)):
vname = name.name;
path = os.path.join(GetVideoSourcesDir(), vname) + os.sep;
if (os.path.isdir(path)):
return VideoSource(path=path);
def LoadVideo(name, max_height=240):
vname = name;
if (isinstance(name, VisBeatExampleVideo)):
vname = name.name;
path = os.path.join(GetVideoSourcesDir(), vname)+os.sep;
if(os.path.isdir(path)):
vs = VideoSource(path=path);
v = vs.getVersion(max_height=max_height);
v.load(features_to_load = 'all');
return v;
else:
return None;
def Dancefer(source_video, target,
synch_video_beat=0, synch_audio_beat=0,
beat_offset = 0, leadin = None, nbeats=None,
source_harmonic = None, target_harmonic = None, source_harmonic_offset=None, target_harmonic_offset=None,
force_recompute=None, warp_type = 'quad',
name_tag=None, name_tag_prefix=None, output_path = None,
**kwargs):
"""
:param source_video: video to warp
:param target: music to warp to
:param synch_video_beat: integer specifying a beat (as in the nth beat) from the video to synchronize with synch_audio_beat
:param synch_audio_beat: integer specifying a beat (as in the nth beat) from the video to synchronize with synch_video_beat
:param beat_offset: Lets you offset which beats you want to render. This is mostly for testing different parts of an output.
:param leadin: how many beats before the synch beats to render
:param nbeats: lets you restrict output to rendering n beats
:param source_harmonic: can be None, 'half', or 'double'. 'half' will use every other beat, which you can offset with source_harmonic_offset. 'double' will add an additional beat between every consecutive beat. update - added 'third' for waltzes.
:param target_harmonic: can be None, 'half', or 'double'. 'half' will use every other beat, which you can offset with source_harmonic_offset. 'double' will add an additional beat between every consecutive beat. update - added 'third' for waltzes.
:param source_harmonic_offset: optional offset for harmonic
:param target_harmonic_offset: optional offset for harmonic
:param force_recompute:
:param warp_type:
:param name_tag:
:param name_tag_prefix:
:param output_path:
:param kwargs:
:return:
"""
if((output_path is not None) and (not force_recompute)):
if(os.path.exists(output_path)):
return Video(output_path);
if(isinstance(target, Video)):
target_audio = target.getAudio();
else:
target_audio = target;
synchaudio = synch_audio_beat;
synchvideo = synch_video_beat;
lead_in = leadin;
if(lead_in is None):
lead_in = min(synchaudio, synchvideo);
elif(isinstance(lead_in, str) and lead_in[0]=='<'):
# lead_in = min(synchaudio, synchvideo, int(lead_in));
lead_in = min(synchaudio, int(lead_in));
start_audio_beat = synchaudio-lead_in;
start_video_beat = synchvideo-lead_in;
if(beat_offset and beat_offset>0):
start_audio_beat = start_audio_beat+beat_offset;
start_video_beat = start_video_beat+beat_offset;
print(("Warping {} to {}".format(source_video.getName(), target_audio.getName())));
bitrate = None;
vbeats = source_video.audio.getBeatEvents()
tbeats = target_audio.getBeatEvents()
if(start_video_beat < 0):
if(synchvideo == 0):
vbeats = [vbeats[0].clone()]+vbeats;
vbeats[0].start = vbeats[0].start-(vbeats[2].start-vbeats[1].start);
vbadd = Event.SubdivideIntervals(vbeats[:2], -start_video_beat);
vbeats = vbadd+vbeats[2:];
start_video_beat = 0;
vbeats = vbeats[start_video_beat:];
tbeats = tbeats[start_audio_beat:];
if(source_harmonic=='half'):
vbeats = Event.Half(vbeats, source_harmonic_offset);
elif (source_harmonic == 'third'):
vbeats = Event.Third(vbeats, source_harmonic_offset);
elif(source_harmonic == 'double'):
vbeats = Event.Double(vbeats);
if (target_harmonic == 'half'):
tbeats = Event.Half(tbeats, target_harmonic_offset);
elif (target_harmonic == 'third'):
tbeats = Event.Third(tbeats, target_harmonic_offset);
elif (target_harmonic == 'double'):
tbeats = Event.Double(tbeats);
if(nbeats):
print(("Rendering {} beats of result".format(nbeats)))
if(len(vbeats)>nbeats):
vbeats = vbeats[:nbeats];
print((len(vbeats)))
if(len(tbeats)>nbeats):
tbeats = tbeats[:nbeats];
print((len(tbeats)))
else:
if(vbeats[-1].start0):
start_audio_beat = start_audio_beat+beat_offset;
start_video_beat = start_video_beat+beat_offset;
print(("Warping {} to {}".format(source_video.getName(), target_audio.getName())));
bitrate = None;
vbeats = source_beats;
if(source_beats is None):
vbeats = source_video.getVisualBeats();
tbeats = target_beats;
if(target_beats is None):
tbeats = target_audio.getBeatEvents();
if(start_video_beat < 0):
if(synchvideo == 0):
vbeats = [vbeats[0].clone()]+vbeats;
vbeats[0].start = vbeats[0].start-(vbeats[2].start-vbeats[1].start);
vbadd = Event.SubdivideIntervals(vbeats[:2], -start_video_beat);
vbeats = vbadd+vbeats[2:];
start_video_beat = 0;
vbeats = vbeats[start_video_beat:];
tbeats = tbeats[start_audio_beat:];
if (source_harmonic == 'half'):
vbeats = Event.Half(vbeats, source_harmonic_offset);
elif (source_harmonic == 'third'):
vbeats = Event.Third(vbeats, source_harmonic_offset);
elif (source_harmonic == 'double'):
vbeats = Event.Double(vbeats);
if (target_harmonic == 'half'):
tbeats = Event.Half(tbeats, target_harmonic_offset);
elif (target_harmonic == 'third'):
tbeats = Event.Third(tbeats, target_harmonic_offset);
elif (target_harmonic == 'double'):
tbeats = Event.Double(tbeats);
if(nbeats):
print(("Rendering {} beats of result".format(nbeats)))
if(len(vbeats)>nbeats):
vbeats = vbeats[:nbeats];
print((len(vbeats)))
if(unfold_to_n):
vbeats = Event.UnfoldToN(vbeats, unfold_to_n, momentum=momentum);
if (len(tbeats) > len(vbeats)):
tbeats = tbeats[:len(vbeats)];
if(warp_type is 'weight'):
vbeats = source_video.visualBeatsFromEvents(vbeats);
if(name_tag is None):
name_tag = warp_type+'_sab_'+str(start_audio_beat)+'_svb_'+str(start_video_beat);
if(name_tag_prefix is not None):
name_tag = name_tag+name_tag_prefix;
warp_args = dict(target=target_audio,
source_events=vbeats,
target_events = tbeats,
warp_type=warp_type,
force_recompute=force_recompute,
name_tag = name_tag)
if(bitrate):
warp_args.update(dict(bitrate=bitrate));
warp_args.update(kwargs);
warped_result = source_video.getWarped(**warp_args);
if(output_path):
final_output_path = output_path;
if(os.path.isfile(final_output_path)):
output_filename = os.path.basename(output_path);
name_parts = os.path.splitext(output_filename);
output_filename_base = name_parts[0];
output_directory_path = os.path.dirname(output_path);
if (output_directory_path == ''):
output_directory_path = '.'
output_ext = name_parts[1];
ntry = 1;
tryname = output_filename_base+ '_' + str(ntry);
while (os.path.isfile(os.path.join(output_directory_path, tryname+output_ext)) and ntry<100):
ntry = ntry+1;
tryname = output_filename_base + '_' + str(ntry);
final_output_path = os.path.join(output_directory_path, tryname + output_ext);
shutil.copy2(src=warped_result.getPath(), dst=final_output_path);
n_frames_total = warped_result.num_frames_total;
warp_used = warped_result.getInfo('warp_used');
warped_result_final = Video(path = final_output_path, num_frames_total=n_frames_total);
warped_result_final.setInfo(label='warp_used', value=warp_used);
os.remove(warped_result.getPath())
warped_result = warped_result_final;
return warped_result;
def getVBSegments(self,source_video,
source_beats = None,
search_tempo=None,
search_window=None,
max_height=None,
beat_limit=None,
n_return=None,
unary_weight=None,
binary_weight=None,
break_on_cuts=None,
peak_vars=None):
source = source_video;
if(source_beats is None):
if (peak_vars is not None):
vbeats = source.getFeature('simple_visual_beats', **peak_vars);
else:
vbeats = source.getFeature('simple_visual_beats');
else:
vbeats = source_beats;
#
if (search_tempo is not None):
tempo = search_tempo;
beat_time = np.true_divide(60.0, tempo);
clips = VisualBeat.PullOptimalPaths_Basic(vbeats, target_period=beat_time, unary_weight=unary_weight,
binary_weight=binary_weight, break_on_cuts=break_on_cuts,
window_size=search_window);
else:
clips = VisualBeat.PullOptimalPaths_Autocor(vbeats, unary_weight=unary_weight, binary_weight=binary_weight,
break_on_cuts=break_on_cuts, window_size=search_window);
if (beat_limit is None):
beat_limit = 2;
print(("There were {} candidates".format(len(vbeats))));
nclips = 0;
segments = [];
for S in clips:
if (len(S) > beat_limit):
nclips = nclips + 1;
segments.append(S);
if (n_return is not None):
segments.sort(key=len, reverse=True);
segments = segments[:n_return];
return segments;
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/_dancefer_examples.py
================================================
from .VisBeatExampleVideo import *
from ._music_examples import *
# dances = [];
dances.append(VisBeatExampleVideo(display_name = 'SNSD: HAHAHA', name='snsd_hahaha', url='https://www.youtube.com/watch?v=OxNsx0eMsGU', start_beat=42));
dances.append(VisBeatExampleVideo(display_name = 'Boston Dynamics: Spot the Dancing Robot', name='spot_the_dancing_robot', url='https://www.youtube.com/watch?v=kHBcVlqpvZ8', start_beat = 9));
dances.append(VisBeatExampleVideo(display_name = 'Michael Jackson: Thriller', name = 'thriller', url = 'https://www.youtube.com/watch?v=wlHUJbl-t7o', start_beat = 4))
dances.append(VisBeatExampleVideo(display_name = 'LMFAO: Party Rock Anthem',name='party_rock', url='https://www.youtube.com/watch?v=WZRMpC_wh0M', start_beat=4, leadin=4))
dances.append(VisBeatExampleVideo(display_name = 'BTS: Boy in Luv', name='boy_in_luv', url='https://www.youtube.com/watch?v=gqz6Adx63w8', start_beat=34));
dances.append(VisBeatExampleVideo(display_name = 'Thays Monaro: Redneck Woman',name='gwilson_redneck_woman', url='https://www.youtube.com/watch?v=1apy8YPygKo', start_beat=65))
dances.append(VisBeatExampleVideo(display_name = 'Music Express: Dancing in the Street', name='dance_cc_dancing_in_the_street', url='https://www.youtube.com/watch?v=fzZ2CoL3IMQ', start_beat=3));
dances.append(VisBeatExampleVideo(display_name = 'Barney and Friends: Mr. Golden Sun',name='barney_mr_golden_sun', url='https://www.youtube.com/watch?v=ya4yyg9XiI4', start_beat=24))
dances.append(VisBeatExampleVideo(display_name = 'M. Express: Friends Forever', name='dance_cc_friends_forever', url='https://www.youtube.com/watch?v=T6ZX6wkJ5Ns', start_beat=2));
dances.append(VisBeatExampleVideo(display_name = 'Just Dance Kids: Gummy Bear',name='jdk_gummy_bear', url='https://www.youtube.com/watch?v=KVE-T2_vLpY', start_beat=60))
dances.append(VisBeatExampleVideo(display_name = 'M. Express: Feelin Good',name='dance_cc_feelin_good', url='https://www.youtube.com/watch?v=uBWp9rr6w08', start_beat=49));
# dances.append(VisBeatExampleVideo(display_name = 'Kiesza: Hideaway', name='hideaway', url='https://www.youtube.com/watch?v=Vnoz5uBEWOA', start_beat=124))
dances.append(VisBeatExampleVideo(display_name = 'SNSD: Genie', name='snsd_genie', url='https://www.youtube.com/watch?v=6SwiSpudKWI', start_beat=141));
dances.append(VisBeatExampleVideo(display_name = 'Momoland Boom Boom', name='momoland_boom_boom', url='https://www.youtube.com/watch?v=0HKfjsM2hSw', start_beat = 145));#33#
dances.append(VisBeatExampleVideo(display_name = 'Snap: Rhythm is a Dancer', name='rhythm_is_a_dancer', url='https://www.youtube.com/watch?v=fDWFVI8PQOI', start_beat=65));
dances.append(VisBeatExampleVideo(display_name = 'Janet Jackson: Rhythm Nation', name='rhythm_nation', url='https://www.youtube.com/watch?v=OAwaNWGLM0c', start_beat = 226));
dances.append(VisBeatExampleVideo(display_name = 'BTS Just One Day', name='bts_just_one_day', url='https://www.youtube.com/watch?v=KQeAg45p2UI', start_beat=200));
# dances.append(VisBeatExampleVideo(display_name = '', name='', url='', start_beat = ));
# dances.append(VisBeatExampleVideo(display_name = '', name='', url='', start_beat = ));
# dances.append(VisBeatExampleVideo(display_name = '', name='', url='', start_beat = ));
# dances.append(VisBeatExampleVideo(display_name = '', name='', url='', start_beat = ));
# dances.append(VisBeatExampleVideo(display_name = '', name='', url='', start_beat = ));
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/_dancify_examples.py
================================================
from .VisBeatExampleVideo import *
from ._music_examples import *
dances.append(VisBeatExampleVideo(display_name = 'Turtle', name='turtle', url='https://www.youtube.com/watch?v=PWD4gktEUAY', start_beat=0));
dances.append(VisBeatExampleVideo(display_name = 'Louie The Cat', name='louie_the_cat', url='https://www.youtube.com/watch?v=hqFG6d86ygI', start_beat=None));
accidental_dances = [];
accidental_dances.append(VisBeatExampleVideo(display_name="Trump IACP Full", name="trump_iacp_full", url='https://www.youtube.com/watch?v=pu4dPTi6SEg', target_n_beats = 7))
accidental_dances.append(VisBeatExampleVideo(display_name="Trump Tax Reform WH.gov Full", name="trump_taxreform_remarks_1", url="https://www.youtube.com/watch?v=zB-lhTEQdKY"))
# accidental_dances.append(VisBeatExampleVideo(display_name="", name="", url=""))
synth = [];
synth.append(VisBeatExampleVideo(name='synth_ball_metronome_w_noise', url = 'https://youtu.be/-mqJIGdro6A', start_beat = None));
synth.append(VisBeatExampleVideo(name='synth_test_ball_1', url = 'https://youtu.be/SjXsMYBJEF8', start_beat = None));
synth.append(VisBeatExampleVideo(name='synth_test_ball_1_gt_impact_sounds', url = 'https://youtu.be/tsdsQm4iTNg', start_beat = None));
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/_mediafiles.py
================================================
import os
VB_MEDIA_UTILS_PATH = os.path.abspath(__file__)
VB_MEDIA_UTILS_DIR = os.path.abspath(os.path.dirname(__file__));
MEDIAFILES_DIR = os.path.join(VB_MEDIA_UTILS_DIR, 'assets'+os.sep)
AUDIO_FILES_DIR = os.path.join(MEDIAFILES_DIR, 'audio'+os.sep);
AUDIO_FILES = [];
AUDIO_FILE_PATHS = {};
for filename in os.listdir(AUDIO_FILES_DIR):
# if(reduce(lambda x,y: x or y, map(lambda ext: filename.lower().endswith(ext), Audio.MEDIA_FILE_EXTENSIONS()))):
AUDIO_FILES.append(filename);
AUDIO_FILE_PATHS[filename]=(os.path.join(AUDIO_FILES_DIR, filename));
def GetTestAudioPath(filename):
return AUDIO_FILE_PATHS[filename];
VIDEO_FILES_DIR = os.path.join(MEDIAFILES_DIR, 'video'+os.sep);
VIDEO_FILES = [];
VIDEO_FILE_PATHS = [];
if(os.path.exists(VIDEO_FILES_DIR)):
for filename in os.listdir(VIDEO_FILES_DIR):
VIDEO_FILES.append(filename);
VIDEO_FILE_PATHS.append(os.path.join(VIDEO_FILES_DIR, filename));
IMAGE_FILES_DIR = os.path.join(MEDIAFILES_DIR, 'images'+os.sep);
IMAGE_FILES = [];
IMAGE_FILE_PATHS = {};
for filename in os.listdir(IMAGE_FILES_DIR):
IMAGE_FILES.append(filename);
IMAGE_FILE_PATHS[filename] = (os.path.join(IMAGE_FILES_DIR, filename));
def GetTestImagePath(filename=None):
if(filename is None):
filename = "VisBeatWatermark.png"
return IMAGE_FILE_PATHS[filename];
def GetVBMarkPath():
return IMAGE_FILE_PATHS["VisBeatWatermark.png"];
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/_music_examples.py
================================================
from .VisBeatExampleVideo import *
dances = [];
music = [];
music.append(VisBeatExampleVideo(display_name = 'U Cant Touch This', name='cant_touch_this', url='https://www.youtube.com/watch?v=_NNYI8VbFyY', start_beat=2, leadin=2));
music.append(VisBeatExampleVideo(display_name = 'Mary Poppins: Supercalifrag.', name='supercalifrag', url='https://www.youtube.com/watch?v=rihNRTTcztQ', start_beat = 5, leadin = 3));
music.append(VisBeatExampleVideo(display_name = 'Migos: Walk It Talk It',name='walkit_talkit', url='https://www.youtube.com/watch?v=fGqdIPer-ms', start_beat = 134));
music.append(VisBeatExampleVideo(display_name = 'Rammstein: Du Hast',name='du_hast', url = 'https://www.youtube.com/watch?v=W3q8Od5qJio', start_beat = 193));
music.append(VisBeatExampleVideo(display_name = 'Taylor Swift: Shake It Off', name='shake_it_off', url='https://www.youtube.com/watch?v=nfWlot6h_JM', start_beat=112));
music.append(VisBeatExampleVideo(display_name = 'Michael Jackson: Billie Jean', name='billie_jean', url='https://www.youtube.com/watch?v=Zi_XLOBDo_Y', start_beat=169));
music.append(VisBeatExampleVideo(display_name = 'Hall and Oates: You Make My Dreams', name='you_make_my_dreams', url='https://www.youtube.com/watch?v=EErSKhC0CZs', start_beat=95));
music.append(VisBeatExampleVideo(display_name = 'Parry Gripp: Breakfast Burrito',name="breakfast_burrito", url='https://www.youtube.com/watch?v=prPjpwsGiws', start_beat=32))
music.append(VisBeatExampleVideo(display_name = 'Pharrell Williams: Happy',name='happy', url='https://www.youtube.com/watch?v=y6Sxv-sUYtM', start_beat=69, leadin=3))
music.append(VisBeatExampleVideo(display_name = 'Aqua: Barbie Girl',name='barbie_girl', url='https://www.youtube.com/watch?v=ZyhrYis509A', start_beat=67, leadin=4));
music.append(VisBeatExampleVideo(display_name = 'Jhameel: Montage',name='music_cc_jhameel_montage', url='https://www.youtube.com/watch?v=vYr_nCmsgoE', start_beat=103))
music.append(VisBeatExampleVideo(display_name = 'Vengaboys: We like to Party',name='we_like_to_party', url='https://www.youtube.com/watch?v=6Zbi0XmGtMw', start_beat=65))
music.append(VisBeatExampleVideo(display_name = 'Rick Astley: Never Gonna Give You Up', name='rick_roll', url='https://www.youtube.com/watch?v=dQw4w9WgXcQ', start_beat = 3, leadin=3));
music.append(VisBeatExampleVideo(display_name = 'Mark Ronson: Uptown Funk ft. Bruno Mars', name='uptown_funk', url='https://www.youtube.com/watch?v=OPf0YbXqDm0', start_beat=32));
music.append(VisBeatExampleVideo(display_name = 'Missy Elliot: Work It', name='work_it', url='https://www.youtube.com/watch?v=cjIvu7e6Wq8', start_beat=52, leadin = 1));
music.append(VisBeatExampleVideo(display_name = 'If I only had a brain', name='only_had_a_brain', url='https://www.youtube.com/watch?v=nauLgZISozs', start_beat=14));
music.append(VisBeatExampleVideo(display_name = 'Kane Brown: Lose It', name='kb_lose_it', url='https://www.youtube.com/watch?v=Z8nkc2KKjd8', start_beat=56));
# music.append(VisBeatExampleVideo(display_name = 'Check Meowt', name='pg_check_meowt', url='https://youtu.be/NjKYX4bQpf0', start_beat = 16));
# music.append(VisBeatExampleVideo(display_name = 'Wu-Tang Clan: Protect Ya Neck', name='wutang_protect_ya_neck', url='https://www.youtube.com/watch?v=R0IUR4gkPIE', start_beat = None));
# music.append(VisBeatExampleVideo(display_name = 'Ice Cube: It Was A Good Day',name='it_was_a_good_day', url='https://www.youtube.com/watch?v=h4UqMyldS7Q', start_beat=64, harmonic = 'half'));
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/command_line.py
================================================
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/fileui/__init__.py
================================================
from sys import platform
if platform == "linux" or platform == "linux2":
PLATFORM = 'linux';
elif platform == "darwin":
PLATFORM = 'osx';
elif platform == "win32":
PLATFORM = 'windows'
SUPPORTED = False;
INITIAL_DIR ='./';
if(PLATFORM == 'osx'):
from . import uipath
def GetFilePath(initial_path=None):
if(initial_path is None):
return uipath.uiGetFilePath(initial_path=INITIAL_DIR);
else:
return uipath.uiGetFilePath(initial_path);
def GetDirectory(initial_path=None):
if(initial_path is None):
return uipath.uiGetDirectory(initial_path=INITIAL_DIR);
else:
return uipath.uiGetDirectory(initial_path);
def GetSaveFilePath(initial_path=None, file_extension = None):
if(initial_path is None):
return uipath.uiGetSaveFilePath(initial_path=INITIAL_DIR, file_extension=file_extension);
else:
return uipath.uiGetSaveFilePath(initial_path=initial_path, file_extension=file_extension);
def Show(path):
uipath.showInFinder(path);
def Open(path):
uipath.openOSX(path);
SUPPORTED = True;
else:
def GetFilePath(initial_path=None):
return None;
def GetDirectory(initial_path=None):
return None;
def GetSaveFilePath(initial_path=None, file_extension = None):
return None;
def Show(path):
return None;
def Open(path):
return None;
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/fileui/uipath.py
================================================
import os
import subprocess
def uiGetFilePath(initial_path=None):
try:
if(initial_path):
output = subprocess.check_output("osascript -e 'set strPath to POSIX file \"{}\"' -e 'set theDocument to choose file with prompt \"Please select a document to process:\" default location strPath' -e 'set theDocument to (the POSIX path of theDocument)'".format(initial_path), shell=True)
else:
output = subprocess.check_output("osascript -e 'set theDocument to choose file with prompt \"Please select a document to process:\"' -e 'set theDocument to (the POSIX path of theDocument)'", shell=True)
return output.replace('\n', '');
except subprocess.CalledProcessError as e:
print((e.output));
# assert(False)
# grabpath = get_ipython().run_cell_magic(u'bash', u'', "osascript -e 'set theDocument to choose file with prompt \"Please select a document to process:\"' -e 'set theDocument to (the POSIX path of theDocument)'>&2")
def uiGetDirectory(initial_path=None):
try:
if(initial_path):
output = subprocess.check_output("osascript -e 'set strPath to POSIX file \"{}\"' -e 'set thedir to choose folder with prompt \"Please select a file:\" default location strPath' -e 'set thedir to (the POSIX path of thedir)'".format(initial_path), shell=True)
else:
output = subprocess.check_output("osascript -e 'set thedir to choose folder with prompt \"Please select a directory:\"' -e 'set thedir to (the POSIX path of thedir)'", shell=True)
return output.replace('\n', '');
except subprocess.CalledProcessError as e:
print((e.output));
# assert(False)
def uiGetSaveFilePath(initial_path=None, file_extension=None):
try:
osastr = "osascript ";
if(initial_path):
osastr = osastr+"-e 'set strPath to POSIX file \"{}\"' ".format(initial_path);
osastr = osastr+"-e 'set theDocument to choose file name with prompt \"Save As File:\" ";
if(initial_path):
osastr = osastr+"default location strPath";
osastr = osastr+"' ";
osastr = osastr+"-e 'set theDocument to (the POSIX path of theDocument)'"
output = subprocess.check_output(osastr, shell=True);
ostring = output.replace('\n', '');
if (file_extension is not None):
if (not ostring.endswith(file_extension)):
ostring = ostring + file_extension;
return ostring;
except subprocess.CalledProcessError as e:
AWARN('ERROR')
print((e.output));
def showInFinder(path):
return openOSX(get_dir_from_path(path));
def openOSX(path):
return subprocess.check_output("open {}".format(put_string_in_quotes(path)), shell=True);
def put_string_in_quotes(s):
return "\""+s+"\""
def get_file_name_from_path(pth):
return os.path.split(pth)[1];
def get_dir_from_path(pth):
return (os.path.split(pth)[0]+os.sep);
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/vbgui/BeatGUI.py
================================================
from visbeat3.AImports import *
VIEWER_INSTALLED = 1;
try:
import vbwidget as Viewer
except ImportError as e:
VIEWER_INSTALLED = 0;
AWARN("VBViewer not installed. Consider installing for full functionality.")
from ..TimeSignal import *
from ..EventList import *
#this is what media should call to get its gui object
def media_GUI_func(self):
if (self._gui is None):
self._gui = BeatGUI();
self._gui.media = self;
return self._gui;
class BeatGUI(AObject):
"""
"""
def AOBJECT_TYPE(self):
return 'BeatGUI';
def __init__(self, media=None, path=None, clear_temp=None):
"""If you provide a directory, it will look for a existing AFileManager.json in that directory, or create one if it does not already exist.
If you provide a json, it will use that json, unless the json doesn't exist, in which case it will complain...
"""
AObject.__init__(self, path=path);
if(media is not None):
self.media = media;
def initializeBlank(self):
AObject.initializeBlank(self);
self._widget = None;
self._media = None;
def getJSONName(self):
return self.AOBJECT_TYPE()+".json";
#########
#
@property
def media(self):
return self._getMedia();
def _getMedia(self):
return self._media;
@media.setter
def media(self, value):
self._setMedia(value);
def _setMedia(self, value):
self._media = value;
#
#
@property
def media_type(self):
return self._getMediaType();
def _getMediaType(self):
if(self.media is None):
return None;
else:
return self.media.AObjectType();
#
#
@property
def widget(self):
return self._getWidget();
def _getWidget(self):
if (self._widget is None):
self._widget = Viewer.VBVSignal();
return self._widget;
@widget.setter
def widget(self, value):
self._setWidget(value);
def _setWidget(self, value):
self._widget = value;
#
#
@property
def frame_rate(self):
return self._getFrameRate();
def _getFrameRate(self):
gfr = self.widget.frame_rate;
if (gfr is None):
media = self.media;
if (media is not None):
gfr = media.getFrameRate();
return gfr;
@frame_rate.setter
def frame_rate(self, value):
self._setFrameRate(value);
def _setFrameRate(self, value):
self.widget.frame_rate = float(value);
#
#
@property
def frame_offset(self):
return self._getFrameOffset();
def _getFrameOffset(self):
return self.widget.frame_offset;
@frame_offset.setter
def frame_offset(self, value):
self._setFrameOffset(value);
def _setFrameOffset(self, value):
self.widget.frame_offset = value;
#
def run(self, local_saliency=None, frame_rate = None, eventlist = 'default', frame_offset=None):
if(frame_rate is None):
# self.widget.frame_rate = float(self.getMedia().getFrameRate());
self.frame_rate = self.media._getFrameRate();
else:
# self.widget.frame_rate = float(frame_rate);
self.frame_rate = frame_rate;
if(local_saliency is None):
self.widget.signal = self.media.getLocalRhythmicSaliency().tolist();
# self.widget.signal = self.getBothWayVisualImpactEnvelope(highpass_window_seconds=None, force_recompute = True).tolist();
else:
self.widget.signal = local_saliency.tolist();
if(frame_offset is None):
self.frame_offset = 0;
elif(frame_offset is 'guess'):
self.frame_offset = self.guessFrameOffset();
else:
self.frame_offset = frame_offset;
if(eventlist is None):
self.widget.events = [];
elif(eventlist == 'default'):
self.widget.events = EventList._toGUIDicts(self.media.getEventList());
else:
self.widget.events = EventList._toGUIDicts(eventlist);
self.widget.data_string = self.media.getStringForHTMLStreamingBase64();
return self.widget;
def guessFrameOffset(self):
if(isinstance(self.media, Video)):
return self.media.reader.get_length() - self.media.n_frames();
else:
return 0;
def deactivateAllEvents(self):
newes = []
gevents = self.getEventDicts();
for e in gevents:
newe = e;
newe['is_active']=0;
newes.append(newe);
self.widget.events = [];
self.widget.events = newes;
def activateAllEvents(self):
newes = []
gevents = self.getEventDicts();
for e in gevents:
newe = e;
newe['is_active'] = 1;
newes.append(newe);
self.widget.events = [];
self.widget.events = newes;
def activatePattern(self, pattern=None, prefix=None, apply_to_active=None):
assert(pattern), "must provide pattern to activate"
newes = []
gevents = self.getGUIEventDicts();
counter = 0;
prefix_length = 0;
if(prefix_length is not None):
prefix_length = len(prefix);
for i, e in enumerate(gevents):
if (apply_to_active):
if (e.get('is_active')):
if (counter < prefix_length):
e['is_active']=prefix[counter];
else:
e['is_active'] = pattern[(counter - prefix_length) % len(pattern)];
counter = counter + 1;
else:
print(("Skipping beat {}, inactive".format(i)));
else:
if (i < prefix_length):
e['is_active'] = prefix[i];
else:
e['is_active'] = pattern[(i - prefix_length) % len(pattern)];
newes.append(e);
self.widget.events = [];
self.widget.events = newes;
def shiftEventsByNFrames(self, n_frames=None):
assert(n_frames), "must provide number of frames to shift by"
newes = []
gevents = self.getEventDicts();
sample_step = np.true_divide(1.0,self.getFrameRate());
for e in gevents:
newe = e;
newe['start'] = newe['start']+n_frames*sample_step;
newes.append(newe);
self.widget.events = [];
self.widget.events = newes;
def getActiveEventTimes(self):
gevents = self.getEventDicts(active_only=True);
revents = []
for e in gevents:
revents.append(e.get('time'));
return np.asarray(revents);
def getEventTimes(self):
gevents = self.getEventDicts();
revents = []
for e in gevents:
revents.append(e.t);
return np.asarray(revents);
def getEvents(self, active_only=None):
return Event._FromGUIDicts(self.getEventDicts(active_only = active_only));
def getEventList(self, active_only=None):
elist = EventList._FromGUIDicts(self.getEventDicts(active_only=active_only));
elist.setInfo(label='html_frame_offset', value=self.getFrameOffset());
return elist;
def getActiveEvents(self):
return self.getEvents(active_only=True);
def getEventDicts(self, active_only = None):
gevents = self.widget.events[:];
if(not active_only):
return gevents;
else:
nevents = []
for e in gevents:
if(e.get('is_active')):
nevents.append(e);
return nevents;
def saveEvents(self, save_path = None):
elist = self.getEventList(active_only=False);
if(save_path is not None):
elist.writeToJSON(json_path=save_path);
self.widget.last_save_path = save_path;
else:
save_path = self.widget.last_save_path;
if(save_path is None):
save_path = uiGetSaveFilePath(file_extension='.json');
if(save_path is not None):
elist.writeToJSON(json_path=save_path);
self.widget.last_save_path = save_path;
def saveEventsAs(self, save_path = None):
elist = self.getEventList(active_only=False);
if (save_path is not None):
elist.writeToJSON(json_path=save_path);
self.widget.last_save_path = save_path;
print(('savepath not none {}'.format(save_path)))
else:
save_path = uiGetSaveFilePath(file_extension='.json');
print(('savepath from ui {}'.format(save_path)))
if (save_path is not None):
print(('save path from ui {}'.format(save_path)));
elist.writeToJSON(json_path=save_path);
self.widget.last_save_path = save_path;
print(save_path)
def setEvents(self, events):
self.widget.events = Event._ToGUIDicts(events);
def setEventList(self, event_list):
if(event_list.getInfo('html_frame_offset') is not None):
self.widget.frame_offset = event_list.getInfo('html_frame_offset');
self.widget.events = event_list._toGUIDicts();
def loadEvents(self, load_path = None):
if(load_path is None):
load_path = uiGetFilePath();
elist = EventList();
elist.loadFromJSON(json_path=load_path);
self.setEventList(event_list = elist);
def getEventListWithSelectedSegments(self):
eventlist = self.getEventList();
events = eventlist.events;
segments = [];
for i, e in enumerate(events):
if(e.direction>-1): # meaning not a back beat
newseg = [];
for si in range(i, len(events)):
newseg.append(si);
if(events[si].direction<0): # meaning a back beat
break;
segments.append(newseg);
eventlist.setInfo(label='selected_segments', value=segments);
return eventlist;
================================================
FILE: src/video2npz/visbeat3/build/lib/visbeat3/vbgui/__init__.py
================================================
================================================
FILE: src/video2npz/visbeat3/build/scripts-3.7/dancefer
================================================
#!/opt/anaconda3/envs/py37/bin/python
import sys
import matplotlib
matplotlib.use('PS')
import visbeat3
source_url = 'https://www.youtube.com/watch?v={}'.format(sys.argv[0]);
target_url = 'https://www.youtube.com/watch?v={}'.format(sys.argv[1]);
output_path = sys.argv[2];
result = visbeat3.AutoDancefer(source=source_url, target = target_url,
output_path = output_path,
synch_video_beat = 0,
synch_audio_beat = 0,
beat_offset = 64,
nbeats = 128);
result.play();
================================================
FILE: src/video2npz/visbeat3/setup.cfg
================================================
[metadata]
# This includes the license file(s) in the wheel.
# https://wheel.readthedocs.io/en/stable/user_guide.html#including-license-files-in-the-generated-wheel-file
license_files = LICENSE
[bdist_wheel]
# This flag says to generate wheels that support both Python 2 and Python
# 3. If your code will not run unchanged on both Python 2 and 3, you will
# need to generate separate wheels for each Python version that you
# support. Removing this line (or setting universal to 0) will prevent
# bdist_wheel from trying to make a universal wheel. For more see:
# https://packaging.python.org/guides/distributing-packages-using-setuptools/#wheels
universal=0
================================================
FILE: src/video2npz/visbeat3/setup.py
================================================
import setuptools
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="visbeat3",
version="0.0.8",
author="Haofan Wang",
author_email="haofanwang.ai@gmail.com",
description="Python3 Implementation for 'Visual Rhythm and Beat' SIGGRAPH 2018",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/haofanwang/visbeat3",
packages=setuptools.find_packages(),
# install_requires=[
# 'numpy',
# 'scipy',
# 'bs4',
# 'librosa==0.6.0',
# 'imageio==2.9.0',
# 'requests',
# 'moviepy==1.0.3',
# 'termcolor',
# 'youtube-dl',
# 'matplotlib',
# 'numba==0.48.0'
# ],
scripts=['bin/dancefer'],
include_package_data=True,
package_data={'data': ['visbeat3/assets/*']},
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: Apache Software License",
"Operating System :: OS Independent",
],
)
================================================
FILE: src/video2npz/visbeat3/test.py
================================================
import visbeat3 as vb
import os
video_dir = '../videos/'
tempos = {}
# try:
# for file in os.listdir(video_dir):
# if file == '.DS_Store':
# continue
# vlog = vb.PullVideo(name=file, source_location=video_dir+file, max_height=360)
# # vbeats = vlog.getVisualBeatSequences(search_window=None)[0]
# # print("vbeats are", vbeats)
# vb.Video.getVisualTempo = vb.Video_CV.getVisualTempo
# tempo = vlog.getVisualTempo()
# print(file, "tempo", tempo)
# tempos[file] = tempo
# finally:
# print(tempos)
vlog = vb.PullVideo(source_location="../videos/wzk_vlog_beat_enhance1_track1238.mp4", max_height=360)
vb.Video.getVisualTempo = vb.Video_CV.getVisualTempo
tempo = vlog.getVisualTempo()
print(tempo)
# vbeats = vlog.getVisualBeatSequences(search_window=None)[0]
# for vbeat in vbeats:
# print(vbeat.start, vbeat.type, vbeat.weight, vbeat.index, vbeat.unrolled_start, vbeat.direction)
================================================
FILE: src/video2npz/visbeat3/visbeat3/ADefines.py
================================================
# AD_DEBUG = 1;
AD_DEBUG = 0;
================================================
FILE: src/video2npz/visbeat3/visbeat3/AFileManager.py
================================================
from .AObject import *
#import shutil
from distutils.dir_util import copy_tree
class AFileManager(AObject):
"""AFileManager (class): Manages assets. This should really be replaced with a database of some sort...
Attributes:
todo
"""
@staticmethod
def AOBJECT_TYPE():
return 'AFileManager';
def getJSONPath(self):
return self.getPath();
def __init__(self, path=None, clear_temp=None):
"""If you provide a directory, it will look for a existing AFileManager.json in that directory, or create one if it does not already exist.
If you provide a json, it will use that json, unless the json doesn't exist, in which case it will complain...
"""
AObject.__init__(self, path=path);
# self.initializeBlank();
self.initWithPath(path=path, clear_temp=clear_temp);
def initializeBlank(self):
AObject.initializeBlank(self);
self.directories = {};
def getJSONName(self):
return self.AOBJECT_TYPE()+".json";
def initWithPath(self, path=None, clear_temp=None):
oldpath = None
newpath = path;
if(path):
if(os.path.isfile(path)):
self.loadFromJSON(self.getJSONPath()); #assume path property is already set to 'path'
oldpath = self.getPath(); #whatever was in the json, having overwritten path property
elif(os.path.isdir(path)):
json_file_path = path+os.sep+self.getJSONName();
self.setPath(json_file_path);
if(os.path.isfile(self.getJSONPath())):
self.loadFromJSON(json_file_path);
oldpath = self.getPath();
newpath = json_file_path;
# self.setPath(file_path=json_file_path);
else:
newpath=self.getJSONPath()
self.writeToJSON(json_path=newpath);#no json file found, so we create one
else:
assert False, "Given AFileManager path is neither an existing directory or file! path: {} (AFileManager.py)".format(path)
self.setPath(file_path=newpath);
if(oldpath):
oldir = get_dir_from_path(pathstring(oldpath));
newdir = get_dir_from_path(pathstring(newpath));
if(oldir != newdir):
AWARN("FILEMANAGER FOUND FILE MOVED FROM:\n{}\nTO:\n{}\nUPDATING DIRECTORIES...".format(oldir,
newdir));
for d in self.directories:
dpth = self.directories[d];
if(dpth.startswith(oldir)):
dpthst = dpth.lstrip(oldir);
self.directories[d]=os.path.join(newdir,dpthst);
AWARN("{} updated to {}".format(dpth, self.directories[d]));
self.setDir('data', pathstring(self.getDirectoryPath()+os.sep+"Data"+os.sep));
self.setDir('backup', pathstring(self.getDir('data')+"Backups"+os.sep));
self.setDir('temp', pathstring(self.getDir('data')+"TEMP"+os.sep));
temp_dir = self.getDir('temp');
if(os.path.isdir(temp_dir) and (clear_temp)):
for the_file in os.listdir(temp_dir):
file_path = os.path.join(temp_dir, the_file);
try:
if os.path.isfile(file_path):
os.remove(file_path);
#os.unlink(file_path);
#elif os.path.isdir(file_path): shutil.rmtree(file_path)
except Exception as e:
print(e)
make_sure_path_exists(temp_dir);
#Video.VIDEO_TEMP_DIR = temp_dir;
def setDir(self, name, path):
# AWARN("setting {} to {}".format(name, path))
# assert(name is not 'log')
self.directories[name]=path;
make_sure_path_exists(path);
return path;
def addDir(self, name):
assert(name not in self.directories), "tried to add {} dir to AFileManager, but this dir is already set"
return self.setDir(name, pathstring(self.getDirectoryPath()+os.sep+name+os.sep));
def getDir(self, name):
# printDictionary(self.directories)
return self.directories.get(name);
def emptyDir(self, name):
dpth = self.getDir(name);
if(dpth is not None and os.path.isdir(dpth)):
shutil.rmtree(dpth);
make_sure_path_exists(dpth);
def deleteDir(self, name):
dpth = self.getDir(name);
if (dpth is not None and os.path.isdir(dpth)):
shutil.rmtree(dpth);
d = dict(self.directories);
del d[name];
self.directories=d;
def toDictionary(self):
d = AObject.toDictionary(self);
d['directories']=self.directories;
#serialize class specific members
return d;
def copyPathToDir(self, path_to_copy, dest_dir):
dest_path = self.getDir(dest_dir);
if(dest_path):
if(os.path.isdir(path_to_copy)):
copy_tree(src=path_to_copy, dst=dest_path);
elif(os.path.isfile(path_to_copy)):
shutil.copy2(path_to_copy, dest_path)
return;
def copyDirToPath(self, dir_to_copy, dest_path):
src_path = self.getDir(dir_to_copy);
if(src_path):
if(os.path.isdir(dest_path)):
copy_tree(src=src_path, dst=dest_path);
return;
@staticmethod
def copyRandomFractionOfFilesInSourceDir(source_dir, dest_dir, fraction=1.0, ext=None):
"""
Copies a random fraction of files in source directory... Wrote this for splitting training/test data in ML applications.
:param source_dir:
:param dest_dir:
:param fraction:
:param ext:
:return:
"""
directories = []
subdirnames = []
filepaths = [];
for filename in os.listdir(source_dir):
path = os.path.join(source_dir, filename)
if os.path.isdir(path):
directories.append(path)
subdirnames.append(filename)
else:
# namepart, extpart = os.path.splitext(filename);
if((ext is None) or filename.lower().endswith(ext)):
filepaths.append(path);
n_to_copy = int(len(filepaths)*fraction);
random_seed = 0;
random.seed(random_seed);
random.shuffle(filepaths);
copy_sources = filepaths[:n_to_copy];
for src, dst in zip(copy_sources, [dest_dir]*len(copy_sources)):
#print("src: {}\ndst: {}".format(src, dst));
shutil.copy2(src, dst);
for d in range(len(directories)):
subdest = pathstring(os.path.join(dest_dir,subdirnames[d])+os.sep);
make_sure_dir_exists(subdest);
AFileManager.copyRandomFractionOfFilesInSourceDir(source_dir=directories[d], dest_dir=subdest, fraction=fraction, ext=ext);
def initFromDictionary(self, d):
AObject.initFromDictionary(self, d);
self.directories = d['directories'];
def save(self):
if(os.path.isfile(self.getJSONPath())):
os.rename(self.getJSONPath(), self.getDir('backup')+os.sep+self.AOBJECT_TYPE()+".json");
self.writeToJSON(self.getJSONPath());
================================================
FILE: src/video2npz/visbeat3/visbeat3/AFuncDict.py
================================================
from .AParamDict import *
import pickle as pickle
import os
class AFuncDict(AParamDict):
"""AFuncDict (class): Extends AParamDict so that functions can be assigned to features and called whenever
computing those features is necessary.
Attributes:
data: name -> value, params
feature funcs: name -> function for evaluating
"""
def __init__(self, owner=None, name=None, path=None):
AParamDict.__init__(self, owner=owner, name=name, path=path);
self.functions = {};
def getEntry(self, name=None, params=None, force_recompute=False):
d = self.data.get(name);
if((d is not None) and (not force_recompute)):
return d;
else:
f = self.getFunction(name=name);
if(f is not None):
if(params is not None):
if(not params.get('force_recompute')):
params.update(dict(force_recompute=force_recompute));
self.setValue(name=name, value=f(self=self.owner, **params), params=params, modified=True);
else:
self.setValue(name=name, value=f(self=self.owner, force_recompute=force_recompute), params=params, modified=True);
return self.data.get(name);
return None;
def getValue(self, name=None, params=None, force_recompute=False):
d = self.getEntry(name=name, params=params, force_recompute=force_recompute);
if(d is not None):
return d.get('value');
else:
return None;
def getParams(self, name=None):
d = self.data.get(name);
if(d is not None):
return d.get('params');
else:
return None;
def getFunction(self, name=None):
return self.functions.get(name);
def setValue(self, name, value=None, params=None, modified=True):
self.data[name]['value']=value;
self.data[name]['params']=params;
self.setEntryModified(name=name, is_modified=modified)
#self.data[name]['modified']=modified;
def setFunction(self, name, function=None):
self.functions[name]=function;
def saveEntry(self, name, path, force=False):
"""Save one entry to one file."""
if(self.data.get(name) is None):
return None;
if(self.isEntryModified(name=name) or force or (not os.path.isfile(path))):
#pickleToPath
f = open(path, 'wb');
pickle.dump(self.getEntry(name=name), f, protocol=2);
f.close();
self.setEntryModified(name=name, is_modified=False);
#assert(False), "should not be saving in this test";
return True;
def setEntryModified(self, name, is_modified=True):
self.data[name]['modified']=is_modified;
if(is_modified):
self.setModified(is_modified=True);
def isEntryModified(self, name):
entry = self.data.get(name);
if(entry is not None):
m=entry.get('modified');
if(m is not None):
return m;
else:
return True;
else:
assert(False), "checking mod bit on entry that does not exist"
def isModified(self):
return self.modified;
def setModified(self, is_modified):
self.modified=is_modified;
def save(self, path, force=False):
"""save all entries to one file."""
if(force or self.isModified()):
f = open(path, 'wb');
pickle.dump(self.data, f, protocol=2);
f.close();
self.setModified(is_modified=False);
#assert(False), "should not be saving in this test";
return True;
def loadEntry(self, name, path):
"""load one entry from one file."""
f=open(path, 'rb');
self.setEntry(name=name, d=pickle.load(f));
f.close();
self.setEntryModified(name=name, is_modified=False);
return True;
def load(self, path):
"""Load a set of entries all from one file."""
f=open(path, 'rb');
newd = pickle.load(f);
self.data.update(newd);
f.close();
return True;
def getKeyList(self):
return list(self.data.keys());
def getFunctionList(self):
return list(self.functions.keys());
================================================
FILE: src/video2npz/visbeat3/visbeat3/AImports.py
================================================
#AImports
from . import ADefines as defines
import os
import os.path
import errno
import json
import pickle as pickle
import glob
import subprocess
from operator import truediv
import shutil
import time
from time import gmtime, strftime, localtime
import random
from . import fileui
try:
from termcolor import colored
def AWARN(message):
if (defines.AD_DEBUG):
print((colored(message, 'red')))
def AINFORM(message):
if(defines.AD_DEBUG):
print((colored(message, 'blue')))
except ImportError:
if (defines.AD_DEBUG):
print("You do not have termcolor installed (pip install termcolor). AWARN will just show as plain print statements when defines.AD_DEBUG==True...")
def AWARN(message):
if (defines.AD_DEBUG):
print(message);
def AINFORM(message):
if (defines.AD_DEBUG):
print(message);
def local_time_string():
return strftime("%Y-%m-%d_%H:%M:%S", localtime());
def get_temp_file_path(final_file_path="TEMP", temp_dir_path = None):
pparts = os.path.split(final_file_path);
destfolder = pparts[0]+os.sep;
tempdir = temp_dir_path;
if(tempdir is None):
tempdir='.';
destfolder=pathstring(tempdir+os.sep);
tempname = 'TEMP_'+pparts[1];
temptry = 0;
while(os.path.isfile(destfolder+tempname)):
temptry=temptry+1;
tempname = 'TEMP{}_'.format(temptry)+pparts[1];
return pathstring(destfolder+tempname);
def runningInNotebook():
try:
shell = get_ipython().__class__.__name__
if shell == 'ZMQInteractiveShell':
return True # Jupyter notebook or qtconsole
elif shell == 'TerminalInteractiveShell':
return False # Terminal running IPython
else:
return False # Other type (?)
except NameError:
return False # Probably standard Python interpreter
def getshellname():
try:
shell = get_ipython().__class__.__name__
return shell;
except NameError:
return False # Probably standard Python interpreter
def runningInSpyder():
return 'SpyderKernel' in str(get_ipython().kernel.__class__);
def pickleToPath(d, path):
# assert(False)
print("pickling")
f = open(path, 'wb');
pickle.dump(d, f, protocol=2);
f.close();
return True;
def unpickleFromPath(path):
f=open(path, 'rb');
d=pickle.load(f);
f.close();
return d;
def make_sure_path_exists(path):
try:
os.makedirs(path)
except OSError as exception:
if exception.errno != errno.EEXIST:
raise
def make_sure_dir_exists(path):
pparts = os.path.split(path);
destfolder = pparts[0]+os.sep;
try:
os.makedirs(destfolder)
except OSError as exception:
if exception.errno != errno.EEXIST:
raise
def safe_file_name(input_string):
return ''.join([i if ord(i) < 128 else '_' for i in input_string]);
def pathstring(path):
return path.replace(os.sep+os.sep, os.sep);
def is_interactive():
import __main__ as main
return not hasattr(main, '__file__')
def printOb(obj):
for attr in dir(obj):
print("#### Obj.%s = %s\n" % (attr, getattr(obj, attr)))
def pathstring(path):
return path.replace(os.sep+os.sep, os.sep);
def get_prepended_name_file_path(original_file_path, string_to_prepend):
pparts = os.path.split(original_file_path);
destfolder = pparts[0]+os.sep;
pname = string_to_prepend+pparts[1];
return pathstring(destfolder+pname);
def safe_file_name(input_string):
return ''.join([i if ord(i) < 128 else '_' for i in input_string]);
def change_extension(input_path, new_ext):
nameparts = os.path.splitext(input_path);
return nameparts[0]+new_ext;
def printDictionary(obj):
if type(obj) == dict:
for k, v in list(obj.items()):
if hasattr(v, '__iter__'):
print(k)
printDictionary(v)
else:
print('%s : %s' % (k, v))
elif type(obj) == list:
for v in obj:
if hasattr(v, '__iter__'):
printDictionary(v)
else:
print(v)
else:
print(obj)
def spotgt_shift_bit_length(x):
#smallest power of two greater than
return 1<<(x-1).bit_length()
def get_file_name_from_path(pth):
return os.path.split(pth)[1];
def get_dir_from_path(pth):
return (os.path.split(pth)[0]+os.sep);
def get_file_names_from_paths(pths):
r = [];
for p in pths:
r.append(get_file_name_from_path(p));
return r;
def writeDictionaryToJSON(d, json_path=None):
if(json_path):
with open(json_path, 'w') as outfile:
json.dump(d, outfile, sort_keys = True, indent = 4, ensure_ascii=False);
def vtt_to_srt(fileContents):
replacement = re.sub(r'([\d]+)\.([\d]+)', r'\1,\2', fileContents)
replacement = re.sub(r'WEBVTT\n\n', '', replacement)
replacement = re.sub(r'^\d+\n', '', replacement)
replacement = re.sub(r'\n\d+\n', '\n', replacement)
return replacement
================================================
FILE: src/video2npz/visbeat3/visbeat3/AObject.py
================================================
#AO#labels#AObject
import os
import json
from .AParamDict import *
from . import fileui
class AObject(object):
"""AObject (class): This is a paarent class used to implement any comon serialization or typing we might want to do later
Attributes:
labels: dictionary of meta_data
save, load, and clear funcs: these are hooks for functions that manage the object's data on disk.
"""
AOBJECT_BASE_PATH=None;
def __init__(self, path=None, **kwargs):
self.initializeBlank();
if(path):
self.setPath(file_path=path, **kwargs);
def initializeBlank(self):
self.a_info = {'AObjectType': self.AOBJECT_TYPE()};
# self.a_data = AParamDict(owner=self, name='a_data');
self.save_func = None;
self.load_func = None;
self.clear_func = None;
def setPath(self, file_path=None, **kwargs):
if(file_path):
self.a_info['file_path'] = pathstring(file_path);
pparts = os.path.split(self.a_info['file_path']);
self.a_info['file_name'] = pparts[1];
self.a_info['directory_path']=pparts[0];
filename = self.a_info.get('file_name');
if(filename):
nameparts = os.path.splitext(filename);
self.a_info['file_base_name'] = nameparts[0];
self.a_info['file_ext'] = nameparts[1];
self.a_info['base_path'] = kwargs.get('base_path');
if(self.a_info['base_path'] is None):
self.a_info['base_path'] = AObject.AOBJECT_BASE_PATH;
def getPath(self):
if('file_path' in self.a_info):
return self.a_info['file_path'];
else:
return None;
def _showFile(self):
if(fileui.Show is not None):
fileui.Show(self.getPath());
def _open(self):
if(fileui.Open is not None):
fileui.Open(self.getPath());
def getRelativePath(self, base_path=None):
base = base_path;
if(base is None):
base = self.a_info.get('base_path');
if(base is None):
AWARN("Base path not set!");
return str(self.getPath());
def getFileName(self):
if('file_name' in self.a_info):
return self.a_info['file_name'];
else:
return None;
def getFileExtension(self):
if('file_ext' in self.a_info):
return self.a_info['file_ext'];
else:
return None;
def getDirectoryPath(self):
return self.a_info.get('directory_path')
def setInfo(self, label, value):
self.a_info[label]=value;
def getInfo(self, label):
return self.a_info.get(label);
def loadFromJSON(self, json_path=None):
if(json_path):
self.setPath(file_path=json_path);
if('file_path' in self.a_info):
json_text=open(self.a_info['file_path']).read();
d = json.loads(json_text);
self.initFromDictionary(d);
def writeToJSON(self, json_path=None):
#with open(jsonpath+self.name+'.json', 'w') as outfile:
if(not json_path):
json_path = self.a_info.get('file_path');
if(json_path):
with open(json_path, 'w') as outfile:
json.dump(self.toDictionary(), outfile, sort_keys = True, indent = 4, ensure_ascii=False);
def serializeInfo(self):
return self.a_info;
def save(self, features_to_save='all', overwrite=True, **kwargs):
if(self.save_func):
self.save_func(self, features_to_save=features_to_save, overwrite=overwrite, **kwargs);
else:
AWARN("SAVE FUNCTION HAS NOT BEEN PROVIDED FOR {} INSTANCE".format(self.AOBJECT_TYPE()));
def load(self, features_to_load=None, **kwargs):
if(self.load_func):
self.load_func(self, features_to_load=features_to_load, **kwargs);
else:
AWARN("LOAD FUNCTION HAS NOT BEEN PROVIDED FOR {} INSTANCE".format(self.AOBJECT_TYPE()));
########### "VIRTUAL" FUNCTIONS #############
@staticmethod
def AOBJECT_TYPE():
return 'AObject';
def toDictionary(self):
d = {'a_info': self.serializeInfo()};
return d;
def initFromDictionary(self, d):
self.a_info = d.get('a_info');
# ##Example of how these functions should be written in a subclass, here we call the subclass 'AssetManager'
# def AOBJECT_TYPE(self):
# return 'AssetManager';
#
# def toDictionary(self):
# d = AObject.toDictionary(self);
# #serialize class specific members
# return d;
#
# def initFromDictionary(self, d):
# AObject.initFromDictionary(self, d);
# #do class specific inits with d;
================================================
FILE: src/video2npz/visbeat3/visbeat3/AParamDict.py
================================================
from .AImports import *
class AParamDict(object):
"""AParamDict (class): Dictionary that stores values and the parameters used to compute those values. I use this
class for two things. The first is to store parameters for reproducability. The second is to only recompute
values when functions are called with different parameters (some of this latter functionality is tied up in
code that isn't part of the Lite release).
Attributes:
data: name -> value, params
"""
def __init__(self, owner=None, name=None, path=None):
self.name=name;
self.data = {};
self.owner=owner;
self.modified=False;
def getEntry(self, name=None, params=None, force_recompute=False):
d = self.data.get(name);
if((d is not None) and (not force_recompute)):
return d;
return None;
def setEntry(self, name, d):
assert(name!='all' and name!='each'),"Entry named '{}' is reserved in AParamDict".format(name);
self.data[name]=d;
def removeEntry(self, name, assert_if_absent=True, set_modified = True):
if(assert_if_absent):
assert(name in self.data),"Tried to remove entry {} that was not already in {}".format(name, self.__class__);
popentry = self.data.pop(name, None);
if(set_modified):
self.setModified(set_modified);
return popentry;
def hasEntry(self, name=None):
return (self.data.get(name) is not None);
def getValue(self, name=None, params=None, force_recompute=False):
d = self.getEntry(name=name, params=params, force_recompute=force_recompute);
if(d is not None):
return d.get('value');
else:
return None;
def getParams(self, name=None):
d = self.data.get(name);
if(d is not None):
return d.get('params');
else:
return None;
def setValue(self, name, value=None, params=None, modified=True):
self.data[name]['value']=value;
self.data[name]['params']=params;
self.setEntryModified(name=name, is_modified=modified)
def saveEntry(self, name, path, force=False):
"""Save one entry to one file."""
if(self.hasEntry(name=name) is None):
return None;
if(self.isEntryModified(name=name) or force or (not os.path.isfile(path))):
f = open(path, 'wb');
pickle.dump(self.getEntry(name=name), f, protocol=2);
f.close();
self.setEntryModified(name=name, is_modified=False);
return True;
def setEntryModified(self, name, is_modified=True):
self.data[name]['modified']=is_modified;
if(is_modified):
self.setModified(is_modified=True);
def isEntryModified(self, name):
m=self.data[name].get('modified');
if(m is not None):
return m;
else:
return True;
def isModified(self):
return self.modified;
def setModified(self, is_modified):
self.modified=is_modified;
def save(self, path, force=False):
"""save all entries to one file."""
if(force or self.isModified()):
f = open(path, 'wb');
pickle.dump(self.data, f, protocol=2);
f.close();
self.setModified(is_modified=False);
return True;
def loadEntry(self, name, path):
"""load one entry from one file."""
f=open(path, 'rb');
self.setEntry(name=name, d=pickle.load(f));
f.close();
self.setEntryModified(name=name, is_modified=False);
return True;
def load(self, path):
"""Load a set of entries all from one file."""
f=open(path, 'rb');
newd = pickle.load(f);
self.data.update(newd);
f.close();
return True;
def getKeyList(self):
return list(self.data.keys());
================================================
FILE: src/video2npz/visbeat3/visbeat3/Audio.py
================================================
from .EventList import *
from .TimeSignal1D import *
from scipy.io.wavfile import write
import librosa
import librosa.display
from scipy import signal, fftpack
class Audio(TimeSignal1D):
"""Audio (class): A sound, and a bunch of convenience functions to go with it.
Attributes:
x: the sound signal
sampling_rate: the sampling rate
"""
FEATURE_FUNCS = TimeSignal1D.FEATURE_FUNCS.copy();
def __init__(self, path=None, sampling_rate=None, x=None, name=None):
#VBObject.__init__(self, path=path);
TimeSignal1D.__init__(self, path=path, sampling_rate=sampling_rate, x=x);
#self.initializeBlank();
if(name is not None):
self.name = name;
if(path):
self.loadFile();
if(self.name is None):
self.name = self.getInfo('file_name')
#
@property
def name(self):
return self.getName();
def getName(self):
return self._name;
@name.setter
def name(self, value):
self._setName(value);
def _setName(self, value):
self._name = value;
#
def initializeBlank(self):
self.name = None;
TimeSignal1D.initializeBlank(self);
self.n_channels = 1;
def _getFrameRate(self):
return self.getOnsetSamplingRate();
def clone(self):
clone = Audio();
clone.setPath(self.getPath());
clone.x = self.x.copy();
clone.sampling_rate = self.sampling_rate;
clone.n_channels = self.n_channels;
stereo = self.getStereo();
if (stereo is not None):
clone.setInfo('stereo_signal', stereo);
clone.setInfo('stereo_sampling_rate', self.getInfo('stereo_sampling_rate'));
return clone;
def loadFile(self, file_path=None, sampling_rate=None, convert_to_mono=True):
if(file_path):
self.setPath(file_path=file_path);
if('file_path' in self.a_info):
self.x, self.sampling_rate = librosa.load(self.a_info['file_path'],sr=sampling_rate, mono=convert_to_mono);
if(len(self.x.shape)>1):
self.a_info['stereo_signal']=self.x;
self.setInfo('stereo_sampling_rate', self.sampling_rate);
self.x = np.mean(self.x, axis = 0)
print("averaged stereo channels");
def getStereo(self):
return self.a_info.get('stereo_signal');
def getStereoSamplingRate(self):
return self.getInfo('stereo_sampling_rate');
def getStringForHTMLStreamingBase64(self):
encoded = self.getStereoEncodedBase64WAV();
return "data:audio/wav;base64,{0}".format(encoded.decode('ascii'));
def getStereoEncodedBase64WAV(self):
sig = self.getStereo();
sr = self.getStereoSamplingRate();
if (sig is None):
sig = self.getSignal();
sr = self.sampling_rate;
saudio = _make_wav(sig, sr);
encoded = base64.b64encode(saudio);
return encoded;
def getMonoEncodedBase64WAV(self):
sig = self.getSignal();
sr = self.sampling_rate;
saudio = _make_wav(sig, sr);
encoded = base64.b64encode(saudio);
return encoded;
def getLocalRhythmicSaliency(self, **kwargs):
return self.getOnsetEnvelope(**kwargs);
def play(self, autoplay = None):
"""
Play audio. Works in Jupyter. if notebook, audio will play normalized -- this is something html5 audio seems to do by default.
:param autoplay:
:return:
"""
if(ISNOTEBOOK):
# if(normalize):
audiodisp = vb_get_ipython().display.Audio(data=self.getSignal(), rate=self.sampling_rate, autoplay=autoplay);
vb_get_ipython().display.display(audiodisp);
# vb_get_ipython().display.display(vb_get_ipython().display.Audio(data=self.getSignal(), rate=self.sampling_rate, autoplay=False))
# else:
# audiodisp = vb_get_ipython().display.Audio(data=self.getMonoEncodedBase64WAV(), rate=self.sampling_rate,
# autoplay=autoplay);
# vb_get_ipython().display.display(audiodisp);
# print("html render")
# htmlstr = self.getStringForHTMLStreamingBase64()
# ahtml = HTML(data=''''''.format(self.name, htmlstr));
# IPython.display.display(ahtml);
else:
p = pyaudio.PyAudio()
stream = p.open(format=pyaudio.paFloat32,
channels=self.n_channels,
rate=self.sampling_rate,
output=True,
output_device_index=1
)
stream.write(self.getSignal())
stream.stop_stream();
stream.close()
p.terminate()
def playBeats(self, indices=None, beats=None):
beat_inds = indices;
if(beats is None):
beats = self.getBeatEvents();
if(beat_inds is None):
beat_inds = [0, len(beats)-1];
if(not isinstance(beat_inds, list)):
beat_inds=[beat_inds-1, beat_inds, beat_inds+1];
if(beat_inds[0]<0):
start_time = 0;
else:
start_time = beats[beat_inds[0]].start;
if(beat_inds[-1]>len(beats)):
end_time = self.getDuration();
else:
end_time = beats[beat_inds[-1]].start;
self.playSegment(time_range = [start_time, end_time]);
def AudioClipFromBeatRange(self, beat_range, beats=None):
if(beats is None):
beats = self.getBeatEvents();
if(beat_range is None):
beat_range = [0, len(beats)-1];
if(beat_range[1] is None):
beat_range[1]=len(beats)-1;
return self.AudioClip(start=beats[beat_range[0]].start, end=beats[beat_range[1]].start);
def playSegment(self, time_range, autoplay=None):
start_time = time_range[0];
end_time = time_range[1];
if(isinstance(start_time, Event)):
start_time = start_time.start;
if(isinstance(end_time, Event)):
end_time = end_time.start;
if(ISNOTEBOOK):
audiodisp = vb_get_ipython().display.Audio(data=self.getSignalSegment(time_range=[start_time, end_time]), rate=self.sampling_rate, autoplay=autoplay);
vb_get_ipython().display.display(audiodisp);
# vb_get_ipython().display.display(vb_get_ipython().display.Audio(data=self.getSignal(), rate=self.sampling_rate, autoplay=False))
else:
p = pyaudio.PyAudio()
stream = p.open(format=pyaudio.paFloat32,
channels=self.n_channels,
rate=self.sampling_rate,
output=True,
output_device_index=1
)
stream.write(self.getSignalSegment(time_range=[start_time, end_time]))
stream.stop_stream();
stream.close()
p.terminate()
def writeToFile(self, output_path=None, output_sampling_rate=None):
assert(output_path), "must provide path to save audio."
data = self.getSignal();
scaled = np.int16(data/np.max(np.abs(data)) * 32767)
if(output_sampling_rate is None):
output_sampling_rate=44100
write(output_path, output_sampling_rate, scaled)
def setValueRange(self, value_range=None):
if(value_range is None):
value_range = [-1,1];
TimeSignal1D.setValueRange(self, value_range=value_range);
def resample(self, sampling_rate):
new_n_samples = sampling_rate*self.getDuration();
self.x = sp.signal.resample(self.x, int(new_n_samples));
self.sampling_rate = sampling_rate;
def GetResampled(self, sampling_rate):
new_a = self.clone();
new_a.resample(sampling_rate=sampling_rate);
return new_a;
def AlignedTo(self, B):
assert(False),'Abe needs to fix -- broke when messing with alignment code for video';
A = self.clone();
if(A.sampling_rate !=B.sampling_rate):
A.resample(B.sampling_rate);
a_signal = A.getSignal();
b_signal = B.getSignal();
a_duration = self.getDuration();
if(len(a_signal) < len(b_signal)):
siglen = spotgt_shift_bit_length(len(b_signal));
else:
siglen = spotgt_shift_bit_length(len(a_signal));
npada = siglen-len(a_signal);
if(npada>0):
a_signal = np.pad(a_signal, (0,npada), 'constant', constant_values=(0, 0));
npadb = siglen-len(b_signal);
if(npadb>0):
b_signal = np.pad(b_signal, (0,npadb), 'constant', constant_values=(0, 0));
Af = fftpack.fft(a_signal);
Bf = fftpack.fft(b_signal);
Ar = Af.conjugate();
Br = Bf.conjugate();
# Ar = -Af.conjugate();
# Br = -Bf.conjugate();
ashift = np.argmax(np.abs(fftpack.ifft(Ar * Bf)));
print(ashift);
print((np.argmax(np.abs(fftpack.ifft(Af * Br)))));
a_return = Audio();
a_return.n_channels = 1;
a_return.sampling_rate = B.sampling_rate;
a_return.x = np.roll(a_signal, ashift);
return a_return;
def getOffsetFrom(self, B):
return -self.getShiftAmountTo(B);
def getShiftAmountTo(self, B):
"""
Get amount to shift by in seconds (to shift this to alignment with B)
"""
a_signal = self.getSignal();
b_signal = B.getSignal();
a_duration = self.getDuration();
if(self.sampling_rate !=B.sampling_rate):
ansamps = a_duration*B.sampling_rate;
a_signal = sp.signal.resample(a_signal, int(ansamps));
# a_signal = librosa.resample(a_signal, self.sampling_rate, B.sampling_rate, res_type='kaiser_fast');
if(len(a_signal) < len(b_signal)):
siglen = spotgt_shift_bit_length(len(b_signal));
else:
siglen = spotgt_shift_bit_length(len(a_signal));
npada = siglen-len(a_signal);
if(npada>0):
a_signal = np.pad(a_signal, (0,npada), 'constant', constant_values=(0, 0));
npadb = siglen-len(b_signal);
if(npadb>0):
b_signal = np.pad(b_signal, (0,npadb), 'constant', constant_values=(0, 0));
# if(len(a_signal) < len(b_signal)):
# npad = len(b_signal)-len(a_signal);
# a_signal = np.pad(a_signal, (0,npad), 'constant', constant_values=(0, 0));
# print('pad1');
#
# if (len(b_signal) < len(a_signal)):
# npad = len(a_signal) - len(b_signal);
# b_signal = np.pad(b_signal, (0, npad), 'constant', constant_values=(0, 0));
# print('pad2');
# print('point0')
Af = fftpack.fft(a_signal);
Bf = fftpack.fft(b_signal);
Ar = Af.conjugate();
Br = Bf.conjugate();
# Ar = -Af.conjugate();
# Br = -Bf.conjugate();
ashiftab = np.argmax(np.abs(fftpack.ifft(Ar * Bf)));
durationsamples = self.getDuration()*B.sampling_rate;
# if(ashiftab>(durationsamples*0.5)):
# ashiftab = ashiftab-durationsamples;
return truediv(ashiftab, B.sampling_rate);
def getBeatEventList(self, time_range = None):
beats = self.getBeats();
if(time_range is None):
return EventList.FromStartTimes(beats, type='beats');
start_beat = 0;
end_beat = len(beats) - 1;
if(time_range[0] is not None):
while((start_beat0 and (beats[end_beat]>time_range[1])):
end_beat = end_beat-1;
if(end_beat>start_beat):
return EventList.FromStartTimes(beats[start_beat:end_beat], type='beats');
else:
return None;
def getBeatEvents(self, start_time=None, end_time=None):
beat_eventlist = self.getBeatEventList();
return beat_eventlist.events;
def AudioClip(self, start, end):
from . import AudioClip
clip = AudioClip.AudioClip(path=self.getPath(), start=start, end=end);
return clip;
def getWithSoundAdded(self, add_times, sound=None, mute_original=None, gain_original = None):
if(sound is None):
sound = Audio.PingSound(sampling_rate=self.sampling_rate);
if(sound.sampling_rate==self.sampling_rate):
s_toadd = sound.getSignal();
else:
new_n_samples = self.sampling_rate * sound.getDuration();
s_toadd = sp.signal.resample(sound.getSignal(), int(new_n_samples));
result = self.clone();
if(mute_original):
result.x = np.zeros(result.x.shape);
if(gain_original is not None):
result.x = result.x*gain_original;
sl = len(s_toadd);
for t in add_times:
if(t0.005):
nosc = truediv(math.log(noise_floor,0.5),damping);
n_seconds = truediv(nosc, freq);
else:
n_seconds = 10.0;
x = np.sin(np.linspace(0, n_seconds * freq * 2 * np.pi, np.round(n_seconds * sampling_rate)));
dmp = np.linspace(0, n_seconds * freq, np.round(n_seconds * sampling_rate))
dmp = np.power(0.5, dmp * damping);
return np.multiply(x, dmp);
@staticmethod
def PingSound(n_seconds=None, freqs=None, damping=None, sampling_rate = 16000):
if(freqs is None):
# freqs = [400,500,600, 700, 800, 900, 1000, 1100, 1200, 1300];
# freqs = np.arange(4, 25) * 100
freqs = np.arange(5, 25) * 75; # just kind of thought this sounded fine...
if(damping is None):
damping = [0.05]*len(freqs);
if(not isinstance(damping,list)):
damping = [damping]*len(freqs);
s = Audio._getDampedSin(freq=freqs[0], n_seconds = n_seconds, sampling_rate = sampling_rate, damping=damping[0]);
for h in range(1,len(freqs)):
new_s = Audio._getDampedSin(freq=freqs[h], n_seconds = n_seconds, sampling_rate = sampling_rate, damping=damping[h]);
if(len(new_s)>len(s)):
new_s[:len(s)]=new_s[:len(s)]+s;
s = new_s;
else:
s[:len(new_s)] = s[:len(new_s)]+new_s;
sa = Audio(x=s, sampling_rate=sampling_rate, name = 'ping');
# sa.setValueRange(value_range=[-1,1]);
sa.setMaxAbsValue(1.0);
return sa;
##### --- FEATURES --- #####
def getBeats(self, use_full_signal=True, tightness = None, force_recompute=False):
"""
:param use_full_signal: If called from AudioClip class, this will determine whether the full signal is used or
just the clip. This is important because the tempo is a global property. More evidence for a constant tempo
is good, but if the tempo changes over the audio you probably don't want to use the full signal.
:param tightness: How tightly to the tempo are beats picked. Must be positive. 0 would not care about tempo.
100 is default. This is actually part of the penalty used in the dynamic programming objective from
Ellis 2007. In librosa: txwt = -tightness * (np.log(-window / period) ** 2)
:param force_recompute:
:return:
"""
if ((not self.hasFeature(name='beats')) or force_recompute):
beat_args = dict(sr = self.sampling_rate, units = 'time');
if (use_full_signal):
beat_args.update(dict(y = self.getFullSignal()));
else:
beat_args.update(dict(y=self.getSignal()))
if (tightness is not None):
beat_args.update(dict(tightness=tightness));
# print(beat_args)
tempo, beats = librosa.beat.beat_track(**beat_args);
self.setFeature(name='tempo', value=tempo);
self.setFeature(name='beats', value=beats);
return self.getFeature(name='beats');
def getBeatVector(self, vector_length=None, force_recompute=False):
"""use_full_signal only makes a difference in AudioClip subclass."""
if ((not self.hasFeature(name='beatvector')) or force_recompute):
D = self.getDuration();
if (vector_length is None):
vector_length = int(math.ceil(D * 240));
rvec = np.zeros(vector_length);
beats = self.getFeature('beats');
step = truediv(D, vector_length);
for i in range(len(beats)):
b = beats[i];
id = int(truediv(b, step));
rvec[id] = 1;
self.setFeature(name='beatvector', value=rvec);
return self.getFeature(name='beatvector');
def getOnsets(self, use_full_signal=True, force_recompute=False, **kwargs):
"""use_full_signal only makes a difference in AudioClip subclass."""
if ((not self.hasFeature(name='onsets')) or force_recompute):
if (use_full_signal):
onsets = librosa.onset.onset_detect(y=self.getFullSignal(), sr=self.sampling_rate, units='time',
**kwargs);
self.setFeature(name='onsets', value=onsets);
else:
onsets = librosa.onset.onset_detect(y=self.getSignal(), sr=self.sampling_rate, units='time',
**kwargs);
self.setFeature(name='onsets', value=onsets);
return self.getFeature(name='onsets');
def getOnsetSamplingRate(self):
return np.true_divide(self.sampling_rate, AUDIO_DEFAULT_HOP_LENGTH);
def pickOnsets(self, pre_max_time=0.03,
post_max_time=0.0,
pre_avg_time=0.1,
post_avg_time=0.1,
wait_time=0.03,
delta=0.07,
force_recompute=True, **kwargs):
"""
:param pre_max_time:
:param post_max_time:
:param pre_avg_time:
:param post_avg_time:
:param wait_time:
:param force_recompute:
:param kwargs:
:return:
"""
# kwargs.setdefault('pre_max', 0.03 * sr // hop_length) # 30ms
# kwargs.setdefault('post_max', 0.00 * sr // hop_length + 1) # 0ms
# kwargs.setdefault('pre_avg', 0.10 * sr // hop_length) # 100ms
# kwargs.setdefault('post_avg', 0.10 * sr // hop_length + 1) # 100ms
# kwargs.setdefault('wait', 0.03 * sr // hop_length) # 30ms
# kwargs.setdefault('delta', 0.07)
pick_params = dict(
pre_max_time=pre_max_time,
post_max_time=post_max_time,
pre_avg_time=pre_avg_time,
post_avg_time=post_avg_time,
wait_time=wait_time,
delta=delta,
)
tp_keys = list(pick_params.keys());
for p in tp_keys:
pick_params[p] = int(round(self.getOnsetSamplingRate() * pick_params[p]));
dparams = dict(
pre_max=pick_params['pre_max_time'],
post_max=pick_params['post_max_time'] + 1,
pre_avg=pick_params['pre_avg_time'],
post_avg=pick_params['post_avg_time'] + 1,
wait=pick_params['wait_time'],
delta=delta
)
return self.getOnsets(force_recompute=force_recompute, **dparams);
def getEvents(self):
return self.getBeatEvents();
def getEventList(self):
return EventList(self.getBeatEvents());
def getOnsetEvents(self):
onsets = self.getOnsets();
events = Event.FromStartTimes(onsets, type='onsets');
return events;
def getBeatEvents(self, start_time=None, end_time=None, **kwargs):
beats = self.getBeats(**kwargs);
start_beat = 0;
end_beat = len(beats) - 1;
if(start_time is not None):
while((start_beat0 and (beats[end_beat]>end_time)):
end_beat = end_beat-1;
if(end_beat>start_beat):
return Event.FromStartTimes(beats[start_beat:end_beat], type='beats');
else:
return None;
def getOnsetEnvelope(self, use_full_signal=True, force_recompute=False, centering=True, **kwargs):
"""use_full_signal only makes a difference in AudioClip subclass."""
feature_name = 'onset_envelope';
if((not self.hasFeature(name=feature_name)) or force_recompute):
if(use_full_signal):
eval_sig = self.getFullSignal();
else:
eval_sig = self.getSignal();
onsets = librosa.onset.onset_strength(y=eval_sig, sr=self.sampling_rate, centering=centering, hop_length=AUDIO_DEFAULT_HOP_LENGTH, **kwargs);
self.setFeature(name=feature_name, value=onsets);
return self.getFeature(name=feature_name);
def getMelSpectrogram(self, n_mels = 128, force_recompute=False):
feature_name = 'melspectrogram';
if((not self.hasFeature(name=feature_name)) or force_recompute):
params = dict( sr = self.sampling_rate,
n_mels = n_mels);
Spec = librosa.feature.melspectrogram(self.getSignal(), **params);
self.setFeature(name=feature_name, value=librosa.power_to_db(Spec, ref=np.max), params=params);
return self.getFeature(feature_name);
def getSpectrogram(self, hop_length=None, force_recompute=False, **kwargs):
feature_name = 'spectrogram';
if((not self.hasFeature(name=feature_name)) or force_recompute):
# params = dict( sr = self.sampling_rate);
# params.update(kwargs);
params = dict(kwargs);
if(hop_length is None):
hop_length = AUDIO_DEFAULT_HOP_LENGTH;
params['hop_length'] = hop_length;
center = kwargs.get('center');
if(center is None):
center = True;
params['center'] = center;
# print('recomputing {}'.format(hop_length))
S = np.abs(librosa.stft(self.getSignal(), **params));
self.setFeature(name=feature_name, value=S, params=params);
return self.getFeature(feature_name);
def getRMSE(self, force_recompute=False, hop_length=None, frame_length=None):
feature_name = 'rmse';
if((not self.hasFeature(name=feature_name)) or force_recompute):
if(frame_length is None):
frac_of_second = 0.05;
frame_length=max(1, int(self.sampling_rate*frac_of_second));
if(hop_length is None):
hop_length=int(math.floor(frame_length*0.5));
params = dict( hop_length = hop_length,
center = True,
frame_length = frame_length);
rmse = librosa.feature.rmse(y=self.getSignal(), **params);
self.setFeature(name=feature_name, value=np.ndarray.flatten(rmse), params=params);
return self.getFeature(feature_name);
def getBeatTimeBefore(self, t):
return self.getBeatBefore(t=t).start;
def getBeatBefore(self, t):
beats = self.getBeatEvents();
bi = self.getBeatIndexBefore(t=t);
return beats[bi].start;
def getBeatIndexBefore(self, t):
beats = self.getBeatEvents();
for i, b in enumerate(beats):
if(b.start>t):
return i-1;
return len(beats)-1;
def getTempogram(self, window_length=None, force_recompute=None, frame_rate=None, resample_rate = None, **kwargs):
"""
:param self:
:param window_length: in seconds
:param force_recompute:
:param kwargs:
:return:
"""
feature_name = 'tempogram';
if ((not self.hasFeature(feature_name)) or force_recompute):
if (window_length is None):
window_length = DEFAULT_TEMPOGRAM_WINDOW_SECONDS;
tpgparams = {};
tpgparams.update(kwargs);
sr = self.sampling_rate;
y=self.getSignal();
if(resample_rate is None):
resample_rate = 22050;
if(sr>resample_rate):
print(("resampling {}Hz to {}Hz".format(sr, resample_rate)));
y = librosa.core.resample(y, orig_sr=sr, target_sr=resample_rate);
sr = resample_rate;
print("resampled")
if(frame_rate is None):
frame_rate = 30;
hop_length = int(round(truediv(sr,frame_rate)));
win_length = int(round(window_length * frame_rate));
tparams = dict(y=y,
sr=sr,
hop_length=hop_length,
win_length=win_length,
**kwargs);
tpgparams.update(dict(sr=sr,hop_length=hop_length, win_length=win_length));
result = librosa.feature.tempogram(**tparams);
###########
tempo_bpms = librosa.tempo_frequencies(result.shape[0], hop_length=hop_length, sr=sr)
self.setFeature(name='tempogram_bpms', value=tempo_bpms);
self.setFeature(name=feature_name, value=result, params=tpgparams);
self.setInfo(label='tempogram_params',value=tpgparams);
return self.getFeature(feature_name);
def plotTempogram(self, window=None, time_range=None, **kwargs):
tempogram = self.getFeature('tempogram', force_recompute=True);
tparams = self.getInfo('tempogram_params')
toshow = tempogram;
if(window is not None):
wstart=int(round(window[0]*self.sampling_rate));
wend = int(round(window[1]*self.sampling_rate));
toshow = tempogram[:,wstart:wend];
mplt = librosa.display.specshow(toshow, sr=tparams['sr'], hop_length=tparams['hop_length'], x_axis = 'time', y_axis = 'tempo')
plt.legend(frameon=True, framealpha=0.75)
plt.set_cmap('coolwarm')
plt.colorbar(format='%+2.0f dB')
if (time_range is not None):
plt.xlim(time_range);
plt.xlabel('Time (s)')
plt.title('Audio Tempogram');
plt.tight_layout()
return mplt;
def plotOnsets(self, **kwargs):
signal = self.getFeature('onset_envelope');
events = self.getOnsetEvents();
mplt = Event.PlotSignalAndEvents(signal, sampling_rate=self.getOnsetSamplingRate(), events=events, **kwargs);
plt.xlabel('Time (s)')
plt.ylabel('Onset Strength')
return mplt;
def plotBeats(self, **kwargs):
signal = self.getFeature('onset_envelope');
events = self.getBeatEvents();
mplt = Event.PlotSignalAndEvents(signal, sampling_rate=self.getOnsetSamplingRate(), events=events, **kwargs);
plt.title('Onset Envelope and Beats');
plt.xlabel('Time (s)')
plt.ylabel('Onset Strength')
return mplt;
def plotOnsetEnvelope(self, **kwargs):
signal = self.getFeature('onset_envelope');
events = None;
mplt = Event.PlotSignalAndEvents(signal, sampling_rate=self.getOnsetSamplingRate(), events=events, **kwargs);
plt.title('Onset Envelope');
plt.xlabel('Time (s)')
plt.ylabel('Onset Strength')
return mplt;
def plotSignal(self, time_range=None, ylim=None, **kwargs):
signal = self.getSignal();
# Event.PlotSignalAndEvents(signal, sampling_rate=self.getOnsetSamplingRate(), events=events, **kwargs);
times = np.arange(len(signal));
times = times * truediv(1.0, self.sampling_rate);
mplt = plt.plot(times, signal);
if (time_range is not None):
plt.xlim(time_range[0], time_range[1])
if (ylim is not None):
plt.ylim(ylim);
plt.title('Time Signal');
return mplt;
FEATURE_FUNCS['melspectrogram'] = getMelSpectrogram;
FEATURE_FUNCS['spectrogram'] = getSpectrogram;
FEATURE_FUNCS['rmse'] = getRMSE;
FEATURE_FUNCS['beats'] = getBeats;
FEATURE_FUNCS['onsets'] = getOnsets;
FEATURE_FUNCS['beatvector'] = getBeatVector;
FEATURE_FUNCS['tempogram'] = getTempogram;
FEATURE_FUNCS['onset_envelope'] = getOnsetEnvelope;
def _make_wav(data, rate):
""" Transform a numpy array to a PCM bytestring """
import struct
from io import BytesIO
import wave
try:
import numpy as np
data = np.array(data, dtype=float)
if len(data.shape) == 1:
nchan = 1
elif len(data.shape) == 2:
# In wave files,channels are interleaved. E.g.,
# "L1R1L2R2..." for stereo. See
# http://msdn.microsoft.com/en-us/library/windows/hardware/dn653308(v=vs.85).aspx
# for channel ordering
nchan = data.shape[0]
data = data.T.ravel()
else:
raise ValueError('Array audio input must be a 1D or 2D array')
scaled = np.int16(data / np.max(np.abs(data)) * 32767).tolist()
except ImportError:
# check that it is a "1D" list
idata = iter(data) # fails if not an iterable
try:
iter(next(idata))
raise TypeError('Only lists of mono audio are '
'supported if numpy is not installed')
except TypeError:
# this means it's not a nested list, which is what we want
pass
maxabsvalue = float(max([abs(x) for x in data]))
scaled = [int(x / maxabsvalue * 32767) for x in data]
nchan = 1
fp = BytesIO()
waveobj = wave.open(fp, mode='wb')
waveobj.setnchannels(nchan)
waveobj.setframerate(rate)
waveobj.setsampwidth(2)
waveobj.setcomptype('NONE', 'NONE')
waveobj.writeframes(b''.join([struct.pack(' len(self.x)):
self.clipped = np.concatenate((np.zeros(int(endsample - len(self.x))), self.clipped));
def resample(self, sampling_rate):
Audio.resample(self, sampling_rate);
self._pull_clip_potion();
def initializeBlank(self):
Audio.initializeBlank(self);
self.start = None;
self.end = None;
self.resampled = None;
self.clipped = None;
def getSignal(self, resample=False):
if(self.resampled):
return self.resampled;
if(resample):
assert(False), "haven't implemented audio clip resampling yet.";
return self.clipped
================================================
FILE: src/video2npz/visbeat3/visbeat3/Event.py
================================================
from .VisBeatImports import *
class Event(AObject):
"""Event (class): An event in time, either in video or audio
Attributes:
start: when the event starts
"""
DIRECTION_FORWARD = 1;
DIRECTION_BACKWARD = -1;
DIRECTION_BOTH = 0;
_EVENT_PHASE_BASE_RES = 8;
def AOBJECT_TYPE(self):
return 'Event';
def __str__(self):
return str(self.getAttributeDict());
def __init__(self, start=None, type=None, weight=None, index=None, is_active=1, unrolled_start = None, direction=0, **kwargs):
AObject.__init__(self, path=None);
self.start = start;
self.type=type;
self.weight=weight;
self.index = index;
self.unrolled_start = unrolled_start;
self.is_active = is_active;
self.direction = direction;
self.a_info.update(kwargs);
def initializeBlank(self):
AObject.initializeBlank(self);
self.start = None;
self.type = None;
self.weight = None;
self.index = None;
self.unrolled_start = None;
self.is_active = None;
self.direction = None;
def getAttributeDict(self):
d = dict();
d['start'] = self.start;
d['type'] = self.type;
d['weight'] = self.weight;
d['index'] = self.index;
d['unrolled_start'] = self.unrolled_start;
d['is_active'] = self.is_active;
d['direction'] = self.direction;
return d;
def toDictionary(self):
d=AObject.toDictionary(self);
d.update(self.getAttributeDict());
return d;
def initAttributesFromDictionary(self, d):
self.start = d['start'];
self.type = d.get('type');
self.weight = d.get('weight');
self.index = d['index'];
self.unrolled_start = d.get('unrolled_start');
self.is_active = d['is_active'];
if (d.get('direction') is None):
self.direction = 0;
else:
self.direction = d['direction'];
def initFromDictionary(self, d):
AObject.initFromDictionary(self, d);
self.initAttributesFromDictionary(d);
def clone(self, start=None):
newe = Event();
newe.initFromDictionary(self.toDictionary());
if (start):
newe.start = start;
return newe;
def _getIsSelected(self):
return self.getInfo('is_selected');
def _setIsSelected(self, is_selected):
self.setInfo('is_selected', is_selected);
def _getPhase(self, phase_resolution=None):
if (self.getInfo('phase') is None):
return -1;
if (phase_resolution is None):
return self.getInfo('phase');
else:
return self.getInfo('phase') * (phase_resolution / Event._EVENT_PHASE_BASE_RES);
def _setPhase(self, phase, phase_resolution):
if (phase_resolution is None):
self.setInfo('phase', phase);
else:
self.setInfo('phase', phase * (Event._EVENT_PHASE_BASE_RES / phase_resolution));
def _getBoundaryType(self):
return self.getInfo('boundary_type');
def _setBoundaryType(self, boundary_type):
self.setInfo('boundary_type', boundary_type);
@classmethod
def _FromGUIDict(cls, gd, phase_resolution=None):
e = cls();
e.initAttributesFromDictionary(gd);
e._setPhase(gd['phase'], phase_resolution=phase_resolution);
e._setBoundaryType(gd.get('boundary_type'));
e._setIsSelected(gd.get('is_selected'));
return e;
@classmethod
def _FromGUIDicts(cls, gds, type=None):
events = [];
for gd in gds:
ne = cls._FromGUIDict(gd);
if (type is not None):
ne.type = type;
events.append(ne);
return events;
def _toGUIDict(self):
'''
note that is_active is switched to 0 or 1 for javascript/json
:return:
'''
d = self.getAttributeDict();
if(self.is_active):
d['is_active'] = 1;
else:
d['is_active'] = 0;
d['phase'] = self._getPhase();
d['boundary_type'] = self._getBoundaryType();
d['is_selected'] = self._getIsSelected();
return d;
@staticmethod
def _ToGUIDicts(events, active=None):
'''
convert to dicts for javascript GUI
:param events:
:param active: if not none, all of the is_active flags will be set to this
:return:
'''
startind = int(round(time.time() * 1000));
starts = [];
for e in range(len(events)):
de = events[e]._toGUIDict();
if (active is not None):
assert (active is 0 or active is 1), "is_active must be 0 or 1";
de['is_active'] = active;
if (de.get('index') is None):
de['index'] = startind + e;
starts.append(de);
return starts;
def getUnrolledStartTime(self):
if(self.unrolled_start is not None):
return self.unrolled_start;
else:
return self.start;
def getStartTime(self):
return self.start;
def getShifted(self, new_start_time):
return Event(self.start-new_start_time, type=self.type, weight=self.weight, index=self.index);
@staticmethod
def GetUnrolledList(event_list, assert_on_folds=None):
out_list = [];
event0 = event_list[0].clone();
event0.unrolled_start = event0.start;
event0.index = 0;
out_list.append(event0);
for e in range(1,len(event_list)):
newe = event_list[e].clone();
if(assert_on_folds):
assert(newe.start>=out_list[-1].start), 'FOLD (non-monotonic event list) DETECTED WHEN NOT ALLOWED!!!\n see Event.GetUnrolledList'
newe.unrolled_start = out_list[-1].unrolled_start+np.fabs(newe.start-out_list[-1].start);
newe.index = e;
out_list.append(newe);
return out_list;
@staticmethod
def NewFromIndices(event_list, inds):
out_list = [];
for ei in inds:
out_list.append(event_list[ei].clone());
return out_list;
@staticmethod
def RollToNOld(events, n_out, momentum = 0.25):
inds = [0];
step = 1;
n_events = len(events);
for b in range(1,n_out):
lastind = inds[-1];
if ((n_out-b)<(n_events-lastind) or lastind==0):
inds.append(lastind+1);
step = 1;
elif(lastind==(len(events)-1)):
inds.append(len(events)-2);
step = -1;
else:
foreward_p = 0.5+momentum;
roll = np.random.rand();
if(roll-1):
lastf = 0;
for ei in range(1,len(events)):
links.append({});
links[ei]['prev_f']=lastf;
links[ei]['prev_b']=lastb;
lastbu = lastb;
lastfu = lastf;
if(lastbu is None):
lastbu = 0;
if(lastfu is None):
lastfu = 0;
if(events[ei].direction<1):
for lb in range(lastbu,ei):
links[lb]['next_b']=ei;
# print("ei is {} and lastbu is {}, lastb is {}".format(ei, lastbu, lastb));
lastb = ei;
if (events[ei].direction>-1):
for lf in range(lastfu, ei):
links[lf]['next_f'] = ei;
# print("ei is {} and lastfu is {}, lastf is {}".format(ei, lastfu, lastf));
lastf = ei;
return links;
@staticmethod
def RollToN(events, n_out, start_index = 0, momentum=0.1):
links = Event.GetDirectedLinks(events);
inds = [start_index];
step = 1;
n_events = len(events);
for b in range(1, n_out):
lastind = inds[-1];
foreward_p = 0.5 + momentum;
roll = np.random.rand();
if (roll < foreward_p):
step = step;
else:
step = -step;
if(links[lastind].get('prev_b') is None):
step = 1;
if(links[lastind].get('next_f') is None):
step = -1;
if(step>0):
inds.append(links[lastind]['next_f']);
else:
inds.append(links[lastind]['prev_b']);
# print(inds);
return Event.GetUnrolledList(Event.NewFromIndices(events, inds));
@staticmethod
def Clone(event_list):
out_list = [];
for ei in event_list:
out_list.append(ei.clone());
return out_list;
@staticmethod
def SetDirections(event_list, direction):
for e in range(len(event_list)):
event_list[e].direction = direction;
return event_list;
@classmethod
def FromStartTimes(cls, starts, type=None):
events = [];
for s in starts:
events.append(cls(start=s, type=type));
return events;
@classmethod
def FromStartsAndWeights(cls, starts, weights, type=None):
events = [];
assert(len(starts)==len(weights)), 'Event.FromStartsAndWeights got {} starts and {} weights'.format(len(starts), len(weights));
for s in range(len(starts)):
events.append(cls(start=starts[s], weight=weights[s],type=type));
return events;
@staticmethod
def ToStartTimes(events):
starts = np.zeros(len(events));
for e in range(len(events)):
starts[e]=events[e].start;
return starts;
@staticmethod
def ToWeights(events):
weights = np.zeros(len(events));
for e in range(len(events)):
weights[e] = events[e].weight;
return weights;
#endpoint is false if events are already placed at beginning and end
@staticmethod
def RepeatToLength(events, n, endpoints=False):
if(n=0 and newi=0):
T_out.append(target_events[e]);
S_out.append(source_events[s_out_inds[e]]);
return S_out, T_out;
@staticmethod
def Sort(event_list, func=None):
assert(func is None or func=='start' or func=='time'), "have not implemented sort by {}".format(func);
event_list.sort(key=lambda x: x.start);
return event_list;
@staticmethod
def GetSorted(event_list, func=None):
clone = Event.Clone(event_list);
Event.Sort(clone, func=func);
return clone;
@staticmethod
def GetWithTwoWayMerged(event_list, merge_window = 0.1):
event_list_sorted = Event.GetSorted(event_list);
new_events = [];
new_events.append(event_list_sorted[0].clone());
for ei in range(1,len(event_list_sorted)):
thise = event_list_sorted[ei];
if((thise.start-new_events[-1].start)254):
self.data = self.data.astype(np.int);
def nChannels(self):
if(len(self.data.shape)<3):
return 1;
else:
return self.data.shape[2];
def getClone(self):
rimg = Image(path=self.a_info.get('file_path'), data=self.data.copy());
return rimg;
def getGridPixel(self,x,y,repeatEdge=0):
xo = x;
yo = y;
blackedge = np.zeros(1);
if(len(self.data.shape)==3):
blackedge = np.zeros(self.data.shape[2])
if(y>=self.data.shape[0]):
if(repeatEdge==1):
yo = self.data.shape[0]-1
else:
return blackedge
if(y<0):
if(repeatEdge==1):
yo = 0
else:
return blackedge
if(x>=self.data.shape[1]):
if(repeatEdge==1):
xo=self.data.shape[1]-1
else:
return blackedge
if(x<0):
if(repeatEdge==1):
xo = 0
else:
return blackedge
return self.data[yo,xo]
def getPixel(self, x, y, repeatEdge=0):
if(isinstance(y,int) and isinstance(x,int)):
return self.getGridPixel(x,y)
else:
yf = int(np.floor(y))
yc = int(np.ceil(y))
xf = int(np.floor(x))
xc = int(np.ceil(x))
print('getting here?')
print(xf);
print(yf);
tl = self.getGridPixel(xf,yf,repeatEdge)
tr = self.getGridPixel(xc,yf,repeatEdge)
bl = self.getGridPixel(xf,yc,repeatEdge)
br = self.getGridPixel(xc,yc,repeatEdge)
yalpha = y-yf
xalpha = x-xf
topL = tr*xalpha+tl*(1.0-xalpha)
botL = br*xalpha+bl*(1.0-xalpha)
retv = botL*yalpha+topL*(1.0-yalpha)
return retv
def getShape(self):
return np.asarray(self.data.shape)[:];
def getScaled(self, shape=None, shape_xy=None):
shapeis = (shape is not None);
shapexyis = (shape_xy is not None);
assert((shapeis or shapexyis) and not (shapeis and shapexyis)), "Must provide only one of shape or shape_xy for Image.getScaled"
if(shapeis):
sz=[shape[0], shape[1], self.data.shape[2]];
else:
sz=[shape_xy[1],shape_xy[0],self.data.shape[2]];
imK = sp.misc.imresize(self.data, size=sz);
return Image(data=imK);
def getRotated(self, theta):
imR = sp.misc.imrotate(self.data, theta);
return Image(data=imR);
# def splatAtPixCoord(self, im, location=[0,0]):
# self.data[location[0]:(location[0]+im.data.shape[0]), location[1]:(location[1]+im.data.shape[1]),:]=im.data;
def _splatAtPixCoord(self, im, location=[0,0], **kwargs):
is_int = self._is_int;
selftype = self.dtype;
region0 = [location[0], (location[0]+im.data.shape[0])];
region1 = [location[1], (location[1]+im.data.shape[1])];
if(im.n_channels<4):
self.data[region0[0]:region0[1], region1[0]:region1[1],:]=im.data;
return;
if(im.n_channels == 4):
alphamap = np.moveaxis(np.tile(im._pixels_float[:, :, 3], (3, 1, 1)), [0], [2]);
blenda = (im._pixels_float[:, :, :3]) * alphamap + self._pixels_float[region0[0]:region0[1], region1[0]:region1[1],:]*(1.0 - alphamap);
if(is_int):
blenda = (blenda*255).astype(selftype);
self.data[region0[0]:region0[1], region1[0]:region1[1], :] = blenda;
def reflectY(self):
self.data[:,:,:]=self.data[::-1,:,:];
def reflectX(self):
self.data[:,:,:]=self.data[:,::-1,:];
def PIL(self):
return PIM.fromarray(np.uint8(self.data));
def getRGBData(self):
return self.data[:,:,0:3];
def normalize(self, scale=1.0):
self.data = self.data/np.max(self.data.ravel());
self.data = self.data*scale;
def show(self, new_figure = True):
if(ISNOTEBOOK):
if(new_figure):
plt.figure();
if(self.nChannels()==1):
plt.imshow(self.PIL(), cmap='gray');
else:
plt.imshow(self.PIL());#divided by 255
plt.axis('off');
else:
self.PIL().show();
def writeToFile(self, out_path):
self.PIL().save(out_path);
def getEncodedBase64(self):
return base64.b64encode(self.data);
def getDataAsString(self):
return self.data.tostring();
@staticmethod
def FromBase64(encoded_data, shape):
d = base64.decodestring(encoded_data);
npar = np.frombuffer(d, dtype=np.float64);
rIm = Image(data=np.reshape(npar, shape));
return rIm;
@staticmethod
def FromDataString(data_string, shape, dtype=None):
if(dtype is None):
dtype=np.float64;
img_1d = np.fromstring(data_string, dtype=dtype);
reconstructed_img = img_1d.reshape((height, width, -1))
###########################adapted from https://gist.github.com/turicas/1455973##########################
def _get_font_size(self, text, font_path, max_width=None, max_height=None):
if max_width is None and max_height is None:
raise ValueError('You need to pass max_width or max_height')
font_size = 1
text_size = self.get_text_size(font_path, font_size, text)
if (max_width is not None and text_size[0] > max_width) or (max_height is not None and text_size[1] > max_height):
raise ValueError("Text can't be filled in only (%dpx, %dpx)" % text_size)
while True:
if (max_width is not None and text_size[0] >= max_width) or (max_height is not None and text_size[1] >= max_height):
return font_size - 1
font_size += 1
text_size = self.get_text_size(font_path, font_size, text)
def writeOutlinedText(self, xy, text,
font_size=11,
max_width=None,
max_height=None,
encoding='utf8', draw_context = None):
self.writeText(xy=[xy[0]+3, xy[1]+3], text=text, font_filename='RobotoCondensed-Regular.ttf',
font_size=font_size,
max_width=max_width,
color=(0, 0, 0),
max_height=max_height,
encoding=encoding, draw_context=draw_context);
self.writeText(xy=xy, text=text, font_filename='RobotoCondensed-Regular.ttf',
font_size=font_size,
max_width=max_width,
color=(255, 255, 255),
max_height=max_height,
encoding=encoding, draw_context=draw_context);
def writeText(self, xy, text, font_filename='RobotoCondensed-Regular.ttf',
font_size=11,
color=(0, 0, 0),
max_width=None,
max_height=None,
encoding='utf8', draw_context = None):
x=xy[0];
y=xy[1];
font_paths = find_all_files_with_name_under_path(name=font_filename,
path=os.path.dirname(os.path.abspath(__file__)));
font_path = font_paths[0];
if isinstance(text, str):
text = text.decode(encoding)
if font_size == 'fill' and (max_width is not None or max_height is not None):
font_size = self._get_font_size(text, font_path, max_width,
max_height)
text_size = self._get_text_size(font_path, font_size, text)
font = ImageFont.truetype(font=font_path, size=font_size);
# font = ImageFont.truetype(font_filename, font_size)
if x == 'center':
x = (self.data.shape[1] - text_size[0]) / 2
if y == 'center':
y = (self.data.shape[0] - text_size[1]) / 2
if(draw_context is None):
ipil = self.PIL();
draw = ImageDraw.Draw(ipil)
draw.text((x, y), text, font=font, fill=color)
datashape = self.data.shape;
self.data = np.array(ipil.getdata());
self.data.shape = datashape;
else:
draw_context.text((x, y), text, font=font, fill=color);
return text_size
def _get_text_size(self, font_path, font_size, text):
font = ImageFont.truetype(font_path, font_size)
return font.getsize(text)
@staticmethod
def _VBMark():
if(Image._VBMRK is None):
Image._VBMRK = Image(path=GetVBMarkPath());
return Image._VBMRK;
def _vbmark(self):
self._splatAtPixCoord(**self._vbmarker());
def _vbmarker(self):
vbm = Image._VBMark();
wmfrac = min(VBMARK_SIZE * self.width, VBMARK_SIZE * self.height);
scaleval = min(1.0, np.true_divide(wmfrac, vbm.width));
if (scaleval < 1.0):
vbms = vbm.getScaled(shape=[int(vbm.shape[0] * scaleval), int(vbm.shape[1] * scaleval), vbm.shape[2]]);
else:
vbms = vbm.clone();
margin = min(vbm.width, vbm.height, int(VBMMARGIN * self.width), int(VBMMARGIN * self.height));
return dict(im=vbms, location=[self.height - vbms.height - margin, self.width - vbms.width - margin]);
def writeTextBox(self, xy, text, box_width, font_filename='RobotoCondensed-Regular.ttf',
font_size=11, color=(0, 0, 0), place='left',
justify_last_line=False):
x = xy[0];
y = xy[1];
font_paths = find_all_files_with_name_under_path(name=font_filename,
path=os.path.dirname(os.path.abspath(__file__)));
font_path = font_paths[0];
lines = []
line = []
words = text.split()
for word in words:
new_line = ' '.join(line + [word])
size = self._get_text_size(font_path, font_size, new_line)
text_height = size[1]
if size[0] <= box_width:
line.append(word)
else:
lines.append(line)
line = [word]
if line:
lines.append(line)
lines = [' '.join(line) for line in lines if line]
height = y
for index, line in enumerate(lines):
height += text_height
if place == 'left':
self.writeText((x, height), line,
font_filename,
font_size,
color)
elif place == 'right':
total_size = self._get_text_size(font_path, font_size, line)
x_left = x + box_width - total_size[0]
self.writeText((x_left, height), line, font_filename,
font_size, color)
elif place == 'center':
total_size = self._get_text_size(font_path, font_size, line)
x_left = int(x + ((box_width - total_size[0]) / 2))
self.writeText((x_left, height), line,
font_filename,
font_size, color)
elif place == 'justify':
words = line.split()
if (index == len(lines) - 1 and not justify_last_line) or len(words) == 1:
self.writeText((x, height), line,
font_filename,
font_size,
color)
continue
line_without_spaces = ''.join(words)
total_size = self._get_text_size(font_path, font_size,
line_without_spaces)
space_width = (box_width - total_size[0]) / (len(words) - 1.0)
start_x = x
for word in words[:-1]:
self.writeText((start_x, height), word,
font_filename,
font_size, color)
word_size = self._get_text_size(font_path, font_size,
word)
start_x += word_size[0] + space_width
last_word_size = self._get_text_size(font_path, font_size,
words[-1])
last_word_x = x + box_width - last_word_size[0]
self.writeText((last_word_x, height), words[-1],
font_filename,
font_size, color)
return (box_width, height - y)
#####################################################
from . import Image_CV
if(Image_CV.USING_OPENCV):
Image.USING_OPENCV = Image_CV.USING_OPENCV;
Image.RGB2Gray=Image_CV.RGB2Gray;
Image.GrayToRGB=Image_CV.Gray2RGB;
Image.cvGoodFeaturesToTrack=Image_CV.cvGoodFeaturesToTrack;
Image.withFlow=Image_CV.withFlow;
cvDenseFlowFarneback = Image_CV.cvDenseFlowFarneback;
# Image. = Image_CV.
# Image. = Image_CV.
# Image. = Image_CV.
# Image. = Image_CV.
# Image. = Image_CV.
ocv = Image_CV.ocv;
================================================
FILE: src/video2npz/visbeat3/visbeat3/Image_CV.py
================================================
#CVFunctions
from .VisBeatImports import *
import numpy as np
import scipy as sp
from PIL import Image as PIM
#from VBObject import *
from . import Image as vbImage
import math
DEFAULT_FLOW_HISTOGRAM_BINS = 8;
USING_OPENCV=True
try:
import cv2 as ocv
except ImportError:
USING_OPENCV=False;
AWARN('OpenCV not installed; not importing Image_CV')
#these are functions that use OpenCV that aren't class functions
if(USING_OPENCV):
def flow2rgb(flow):
h, w = flow.shape[:2]
fx, fy = flow[:,:,0], flow[:,:,1]
ang = np.arctan2(fy, fx) + np.pi
v = np.sqrt(fx*fx+fy*fy)
hsv = np.zeros((h, w, 3), np.uint8)
hsv[...,0] = ang*(180/np.pi/2)
hsv[...,1] = 255
#v=np.minimum(v*4, 255);
v = np.log(v+1);
vmax = np.max(v[:]);
vf=v/vmax;
v=vf*255;
hsv[...,2] = v;#np.minimum(v*4, 255)
rgb = ocv.cvtColor(hsv, ocv.COLOR_HSV2BGR)
return rgb
def showFlowHSV(flow, new_figure = True):
if(ISNOTEBOOK):
#plt.imshow(self.data*0.0039215686274509);#divided by 255
if(new_figure):
plt.figure();
plt.imshow(PIM.fromarray(flow2rgb(flow)));
plt.axis('off');
else:
self.PIL().show();
def cornerHarris(im, blockSize=None, ksize=None, k=None):
if (blockSize is None):
blockSize = 2;
if (ksize is None):
ksize = 3;
if (k is None):
k = 0.04;
return ocv.cornerHarris(im, 2, 3, 0.04);
def cvDenseFlowFarneback(from_image, to_image, pyr_scale=None, levels=None, winsize=None, iterations=None, poly_n=None, poly_sigma=None, flags=None):
"""Can provide numpy arrays or Image objects as input. Returns the flow from->to."""
# params for Farneback's method
from_im=from_image;
to_im=to_image;
from_is_ob = isinstance(from_image, vbImage.Image);
to_is_ob = isinstance(to_image, vbImage.Image)
if(from_is_ob):
if(from_image.nChannels()>1):
from_im=from_image.getClone();
from_im.RGB2Gray();
from_im=from_im.data;
if(to_is_ob):
if(to_image.nChannels()>1):
to_im=to_image.getClone();
to_im.RGB2Gray();
to_im=to_im.data;
inputs = dict( flow = None,
pyr_scale = pyr_scale,
levels = levels,
winsize = winsize,
iterations = iterations,
poly_n = poly_n,
poly_sigma = poly_sigma,
flags=flags);
default_flow=None;
default_pyr_scale=0.5;
default_levels=int(math.log(float(min(to_im.shape)), 2))-4;
default_winsize = 15;
default_iterations=3;
default_poly_n=5;
default_poly_sigma=1.25;
use_params = dict(flow=default_flow,
pyr_scale=default_pyr_scale,
levels=default_levels,
winsize=default_winsize,
iterations=default_iterations,
poly_n=default_poly_n,
poly_sigma=default_poly_sigma,
flags=0);
for key in inputs:
if(inputs[key] is not None):
use_params[key]=inputs[key];
return ocv.calcOpticalFlowFarneback(prev=from_im, next=to_im, **use_params);
# These functions are to add to Image class
if(USING_OPENCV):
def RGB2Gray(self):
if(self.nChannels()==1):
return;
else:
self.data = ocv.cvtColor(self.data, ocv.COLOR_RGB2GRAY);
def Gray2RGB(self):
if(self.nChannels()==1):
self.data = ocv.cvtColor(self.data, ocv.COLOR_GRAY2RGB)
def cvGoodFeaturesToTrack(self, maxCorners=None, qualityLevel=None, minDistance=None, corners=None, mask=None, blockSize=None, useHarrisDetector=None):
assert(self.data is not None), "must provide image to find features";
if(self.nChannels()==1):
gray_image=self.data.copy();
else:
gray_image=ocv.cvtColor(self.data, cv2.COLOR_RGB2GRAY);
argd={};
if(maxCorners is not None):
d['maxCorners']=maxCorners;
if(qualityLevel is not None):
d['qualityLevel']=qualityLevel;
if(minDistance is not None):
d['minDistance']=minDistance;
if(corners is not None):
d['corners']=corners;
if(mask is not None):
d['mask']=mask;
if(blockSize is not None):
d['blockSize']=blockSize;
if(useHarrisDetector is not None):
d['useHarrisDetector']=useHarrisDetector;
return ocv.goodFeaturesToTrack(gray_image, **argd);
def withFlow(self, flow, step=16):
clone = self.getClone();
if(clone.nChannels()<3):
clone.convertToRGB();
img = clone.data;
h, w = img.shape[:2]
y, x = np.mgrid[step/2:h:step, step/2:w:step].reshape(2,-1).astype(int)
# flow[y, x] is fx, fy
fx, fy = flow[y,x].T
# After vstack, each column: x, y, x+fx, y+fy
# After transpose: each row: x, y, x+fx, y+fy
# After reshape: each row: [[x, y], [x+fx, y+fy]]
lines = np.vstack([x, y, x+fx, y+fy]).T.reshape(-1, 2, 2)
lines = np.int32(lines + 0.5)
ocv.polylines(img, lines, 0, (0, 255, 0), thickness=3)
for (x1, y1), (x2, y2) in lines:
cv2.circle(img, (x1, y1), 1, (0, 255, 0), -1)
return clone
================================================
FILE: src/video2npz/visbeat3/visbeat3/SourceLocationParser.py
================================================
"""
This code was a last minute hack. It works fine enough for parsing youtube urls, but I would use it for any kind of reference.
Based loosely on video backends from django embed video.
"""
import re
import requests
import os
import sys
if sys.version_info.major == 3:
import urllib.parse as urlparse
else:
import urllib.parse
class SourceURL(object):
"""
Modified from django embed video
"""
allow_https = True
is_secure = False
@classmethod
def SourceLocationType(cls):
return cls.__name__;
def __init__(self, source_location):
self.source_location_type = self.SourceLocationType()
self._source_location = source_location
#
@property
def code(self):
"""
unique string to identify file
:return:
"""
return self._getCode();
def _getCode(self):
raise NotImplementedError;
@code.setter
def code(self, value):
self._setCode(value);
def _setCode(self, value):
raise NotImplementedError;
#
@property
def url(self):
"""
URL of video.
"""
return self.get_url()
def get_url(self):
raise NotImplementedError
@property
def protocol(self):
"""
Protocol used to generate URL.
"""
return 'https' if self.allow_https and self.is_secure else 'http'
@property
def thumbnail(self):
"""
URL of video thumbnail.
"""
return self.get_thumbnail_url()
@classmethod
def is_valid(cls, url):
return True if cls.re_detect.match(url) else False
class WebSourceException(Exception):
""" Parental class for all embed_media exceptions """
pass
class VideoDoesntExistException(WebSourceException):
""" Exception thrown if video doesn't exist """
pass
class UnknownBackendException(WebSourceException):
""" Exception thrown if video backend is not recognized. """
pass
class UnknownIdException(VideoDoesntExistException):
"""
Exception thrown if backend is detected, but video ID cannot be parsed.
"""
pass
class YoutubeURL(SourceURL):
"""
for YouTube URLs.
"""
@classmethod
def SourceLocationType(cls):
return 'youtube';
# Compiled regex (:py:func:`re.compile`) to search code in URL.
# Example: ``re.compile(r'myvideo\.com/\?code=(?P\w+)')``
re_code = None
# Compilede regec (:py:func:`re.compile`) to detect, if input URL is valid for current backend.
# Example: ``re.compile(r'^http://myvideo\.com/.*')``
re_detect = None
# Pattern in which the code is inserted.
# Example: ``http://myvideo.com?code=%s``
# :type: str
pattern_url = None
pattern_thumbnail_url = None
re_detect = re.compile(
r'^(http(s)?://)?(www\.|m\.)?youtu(\.?)be(\.com)?/.*', re.I
)
re_code = re.compile(
r'''youtu(\.?)be(\.com)?/ # match youtube's domains
(\#/)? # for mobile urls
(embed/)? # match the embed url syntax
(v/)?
(watch\?v=)? # match the youtube page url
(ytscreeningroom\?v=)?
(feeds/api/videos/)?
(user\S*[^\w\-\s])?
(?P[\w\-]{11})[a-z0-9;:@?&%=+/\$_.-]* # match and extract
''',
re.I | re.X
)
pattern_url = '{protocol}://www.youtube.com/embed/{code}'
pattern_thumbnail_url = '{protocol}://img.youtube.com/vi/{code}/{resolution}'
resolutions = [
'maxresdefault.jpg',
'sddefault.jpg',
'hqdefault.jpg',
'mqdefault.jpg',
]
def get_url(self):
"""
Returns URL folded from :py:data:`pattern_url` and parsed code.
"""
url = self.pattern_url.format(code=self.code, protocol=self.protocol)
url += '?' + self.query.urlencode() if self.query else ''
return url
def get_thumbnail_url(self):
"""
Returns thumbnail URL folded from :py:data:`pattern_thumbnail_url` and
parsed code.
:rtype: str
"""
return self.pattern_thumbnail_url.format(code=self.code,
protocol=self.protocol)
def _getCode(self):
match = self.re_code.search(self._source_location)
if match:
return match.group('code')
parsed_url = urllib.parse.urlparse(self._source_location)
parsed_qs = urllib.parse.parse_qs(parsed_url.query)
if 'v' in parsed_qs:
code = parsed_qs['v'][0]
elif 'video_id' in parsed_qs:
code = parsed_qs['video_id'][0]
else:
raise UnknownIdException('Cannot get ID from `{0}`'.format(self._source_location))
return code
def get_thumbnail_url(self):
"""
Returns thumbnail URL folded from :py:data:`pattern_thumbnail_url` and
parsed code.
:rtype: str
"""
for resolution in self.resolutions:
temp_thumbnail_url = self.pattern_thumbnail_url.format(
code=self.code, protocol=self.protocol, resolution=resolution)
if int(requests.head(temp_thumbnail_url).status_code) < 400:
return temp_thumbnail_url
return None
class FilePathURL(SourceURL):
@classmethod
def SourceLocationType(cls):
return 'file_path';
def __init__(self, source_location):
self.source_location_type = self.SourceLocationType()
self._source_location = source_location;
@classmethod
def is_valid(cls, source_location):
return os.path.isfile(source_location);
def _getCode(self):
name_parts = os.path.splitext(os.path.basename(self._source_location));
return name_parts[0];
SOURCE_LOCATION_TYPES = (
YoutubeURL,
FilePathURL,
)
def ParseSourseLocation(url):
"""
:param url:
:return:
"""
for backend in SOURCE_LOCATION_TYPES:
if backend.is_valid(url):
return backend(url)
raise UnknownBackendException
================================================
FILE: src/video2npz/visbeat3/visbeat3/TimeSignal.py
================================================
from .VBObject import *
from .Event import *
import math
from operator import truediv
from .VisBeatImports import *
class TimeSignal(VBObject):
"""TimeSignal (class): A time signal, and a bunch of convenience functions to go with it.
Attributes:
sampling_rate: the sampling rate
visualizations dictionary of visualizations. (removed - too many dependencies)
"""
def VBBJECT_TYPE(self):
return 'TimeSignal';
def __init__(self, path=None, sampling_rate = None):
VBObject.__init__(self, path=path);
# self.initializeBlank();
# self.visualizations = AFuncDict(owner=self, name='visualizations');
# self.visualizations.functions.update(self.VIS_FUNCS);
if(sampling_rate):
self.sampling_rate=sampling_rate;
def initializeBlank(self):
VBObject.initializeBlank(self);
self.sampling_rate = None;
#
@property
def frame_rate(self):
return self._getFrameRate();
def _getFrameRate(self):
raise NotImplementedError;
#
def getSampleAtTime(self, f):
prev_sample = self.getSampleAtIndex(math.floor(f));
next_sample = self.getSampleAtIndex(math.ceil(f));
sample_progress = f-np.floor(f);
return (next_sample*sample_progress)+(prev_sample*(1.0-sample_progress));
def getSampleAtIndex(self, i):
return self.getTimeForIndex();
def getDuration(self):
assert(False), "getDuration must be implemented for subclass of TimeSignal"
def getSampleDuration(self):
return 1.0/self.sampling_rate;
def getTimeForIndex(self, i):
return i*self.getSampleDuration();
# def toDictionary(self):
# d = VBObject.toDictionary(self);
# assert(False), "haven't implemented toDictionary for {} yet".format(self.VBOBJECT_TYPE())
# #serialize class specific members
# return d;
# def initFromDictionary(self, d):
# VBObject.initFromDictionary(self, d);
# assert(False), "haven't implemented initFromDictionary for {} yet".format(self.VBOBJECT_TYPE())
# #do class specific inits with d;
================================================
FILE: src/video2npz/visbeat3/visbeat3/TimeSignal1D.py
================================================
from .TimeSignal import *
import math
class TimeSignal1D(TimeSignal):
"""TimeSignal1D (class): A time signal, and a bunch of convenience functions to go with it.
Attributes:
sampling_rate: the sampling rate
x: the time signal
"""
def VBOJECT_TYPE(self):
return 'TimeSignal1D';
def __init__(self, path=None, sampling_rate = None, x=None):
TimeSignal.__init__(self, path=path, sampling_rate=sampling_rate);
if(x is not None):
self.x = x;
#self.initializeBlank(); # will be called by parent
def initializeBlank(self):
TimeSignal.initializeBlank(self);#YES KEEP call through weird loops
self.x = None;
def getSignal(self, resampled=False):
return self.x;
def getSignalSegment(self, time_range):
signal = self.getSignal();
seg_start = int(time_range[0]*self.sampling_rate);
seg_end = int(time_range[1]*self.sampling_rate);
return signal[seg_start:seg_end];
def getFullSignal(self):
return self.x;
def getSampleAtTime(self, f):
prev_sample = self.x[int(math.floor(f))];
next_sample = self.x[int(math.ceil(f))];
sample_progress = f-np.floor(f);
return (next_sample*sample_progress)+(prev_sample*(1.0-sample_progress));
def getSampleAtIndex(self, i):
return self.x[i];
def getDuration(self):
return truediv(len(self.getSignal()), self.sampling_rate);
def setValueRange(self, value_range=None):
if(value_range is None):
value_range = [0,1];
data = self.x[:];
currentscale = np.max(data)-np.min(data);
data = (data/currentscale)*(value_range[1]-value_range[0]);
data = data-np.min(data)+value_range[0]
self.x = data;
def setMaxAbsValue(self, max_abs_val=1.0):
data = self.x[:];
currentscale = np.max(np.fabs(data));
data = (data/currentscale)*max_abs_val;
self.x = data;
================================================
FILE: src/video2npz/visbeat3/visbeat3/VBMIDI.py
================================================
# %load VisBeat/MidiParse.py
import mido
import numpy as np
import os
# from TimeSignal1D import *
from .Audio import *
from .Event import *
class VBMIDITrack(object):
def __init__(self, track, ticks_per_beat):
self.mido_track = track;
self.name = track.name;
# print('Track {}: {}'.format(i, track.name));
self.tempo = next((msg.tempo for msg in track if msg.type == 'set_tempo'), None);
self.ticks_per_beat = ticks_per_beat;
# self._get_note_on_times();
self.note_on_times = None;
def getBPM(self):
return mido.tempo2bpm(self.tempo);
def getNoteOnTimes(self, include_negative=None):
if(self.note_on_times is None or (include_negative)):
self._get_note_on_times(include_negative = include_negative);
return self.note_on_times;
def _get_note_on_times(self, include_negative = None):
"""
Gets the starting time of each note.
"""
note_times = []
current_time = 0;
for msg in self.mido_track:
if not msg.is_meta:
delta_time = mido.tick2second(msg.time, self.ticks_per_beat, self.tempo)
current_time += delta_time
if msg.type == 'note_on' and msg.velocity > 0:
if (include_negative or current_time>=0):
note_times.append(current_time);
self.note_on_times = np.array(note_times);
def get_note_durations(self, track_num):
note_durations = []
currently_played_notes = {}
current_time = 0
for msg in self.mido_track:
if not msg.is_meta:
delta_time = mido.tick2second(msg.time, self.ticks_per_beat, self.tempo)
current_time += delta_time
if msg.type == 'note_on' and msg.velocity > 0 and msg.note not in currently_played_notes:
currently_played_notes[msg.note] = current_time
#if len(currently_played_notes) > 1:
# print "Number of played notes have reached: ", len(currently_played_notes)
elif msg.type == 'note_off' or (msg.type == 'note_on' and msg.velocity == 0):
duration = current_time - currently_played_notes[msg.note]
note_durations.append((currently_played_notes[msg.note], duration))
currently_played_notes.pop(msg.note)
assert not bool(currently_played_notes), "Finished looping through all messages. There should not be any notes playing at this point."
#print "before sorting: ", note_durations
note_durations.sort(key=lambda x : x[0])
#print "after sorting: ", note_durations
return np.array([ x[1] for x in note_durations ])
def get_mouth_events(self):
# note_times = []
current_time = 0;
number_on = 0;
events = [];
open_buffer = 0.1;
close_buffer = 0.1
last_type = 0;
events.append(Event(start=0, type='mouth_close', weight=1));
for msg in self.mido_track:
if not msg.is_meta:
delta_time = mido.tick2second(msg.time, self.ticks_per_beat, self.tempo)
current_time += delta_time
last_time = events[-1].start;
passed = current_time-last_time;
if msg.type == 'note_on' and msg.velocity > 0:
if(last_type==0):
if(passed>open_buffer):
events.append(Event(start=current_time-open_buffer, type='mouth_closed', weight=0));
events.append(Event(start=current_time, type='mouth_open', weight = truediv(msg.velocity,127)));
number_on = number_on + 1;
last_type=1;
if(msg.type == 'note_off' or (msg.type=='note_on' and msg.velocity == 0)):
if (last_type == 1):
if (passed > close_buffer):
events.append(Event(start=current_time - close_buffer, type='mouth_opened', weight=0));
events.append(Event(start=current_time, type='mouth_close', weight = truediv(msg.velocity,127)));
number_on = number_on-1;
last_type=0;
# assert(number_on>-1);
# if(number_on==0):
return events;
# self.note_on_times = np.array(note_times);
def getNoteOnTimesAsAudio(self, sampling_rate = None, note_sound=None, n_seconds=None):
assert(n_seconds is not None), "must provide n seconds"
if(sampling_rate is None):
sampling_rate = 16000;
if(note_sound is None):
note_sound = Audio.getPing();
s = Audio.Silence(n_seconds=n_seconds, sampling_rate=sampling_rate, name=self.name)
s = s.getWithSoundAdded(note_sound, self.getNoteOnTimes());
return s;
class VBMIDI(TimeSignal1D):
def __init__(self, path=None):
TimeSignal1D.__init__(self, path=path);
self.midi_file = mido.MidiFile(path)
self.tracks = [];
for i, track in enumerate(self.midi_file.tracks):
self.tracks.append(VBMIDITrack(track, self.midi_file.ticks_per_beat));
def getNoteOnTimes(self, include_negative=None):
return self.tracks[-1].getNoteOnTimes(include_negative=include_negative);
def getMouthEvents(self):
return self.tracks[-1].get_mouth_events();
def getNoteOnTimesAsAudio(self, sampling_rate=None, note_sound=None):
s = self.tracks[-1].getNoteOnTimesAsAudio(sampling_rate = sampling_rate, note_sound = note_sound, n_seconds = self.midi_file.length);
if (s.name is None):
s.name = self.getInfo('file_name')
return s;
================================================
FILE: src/video2npz/visbeat3/visbeat3/VBObject.py
================================================
import os
import json
from .VisBeatImports import *
from .AFuncDict import *
from .AObject import AObject
class VBObject(AObject):
"""VBObject (class): This is a paarent class used to implement common serialization and other functions. There ends
up being three different dictionaries of data.
a_info - for small labels and such. Part of AObject.
a_data - for data to be computed in experiments. I generally use this for things I don't want to automatically
save out to file.
features - these are features tied to the results of functions, and manager classes (e.g. VideoSource) will save
these to disk for future use.
FEATURE_FUNCS is a dictionary mapping the names of features to their corresponding functions.
"""
FEATURE_FUNCS={};
def __init__(self, path=None):
AObject.__init__(self, path=path);
def initializeBlank(self):
AObject.initializeBlank(self);
self.a_info.update({'VBObjectType': self.VBOBJECT_TYPE()});
self.features = AFuncDict(owner=self, name='features');
self.features.functions.update(self.FEATURE_FUNCS);
def saveFeature(self, name, path):
"""Subclasses can implement version of this that will check members for features if those features arent found here."""
return self.features.saveEntry(name=name, path=path);
def saveFeatures(self, path):
return self.features.save(path=path);
def loadFeature(self, name, path):
"""Subclasses can implement version of this that will check whether feature is registered before loading."""
return self.features.loadEntry(name=name, path=path);
def loadFeatures(self, path):
return self.features.load(path=path);
def getFeature(self,name, force_recompute=False, **kwargs):
"""Understood to get the value of a feature. can automatically recompute if feature has registered function."""
params = kwargs;
assert (not kwargs.get('params')), "STILL TRYING TO USE PARAMS INSTEAD OF KWARGS. FIX THIS";
return self.features.getValue(name=name, params=kwargs, force_recompute=force_recompute);
def getFeatureEntry(self, name, params=None, force_recompute=False):
return self.features.getEntry(name=name, params=params, force_recompute=force_recompute);
def getFeatureParams(self, name):
return self.features.getParams(name=name);
def setFeature(self, name, value, params=None):
rval = self.features.setEntry(name=name, d=dict(value=value, params=params));
self.features.setEntryModified(name=name, is_modified=True);
return rval;
def removeFeature(self, name, assert_if_absent=True, set_modified=True):
self.features.removeEntry(name=name, assert_if_absent=assert_if_absent, set_modified=set_modified);
def hasFeature(self, name):
"""Just checks to see if it's there."""
return self.features.hasEntry(name=name);
def getFeatureFunction(self, feature_name):
return self.features.getFunction(name=feature_name);
def getFeaturesList(self):
return self.features.getKeyList();
def getFeatureFunctionsList(self):
return self.features.getFunctionList();
def clearFeatureFiles(self, features_to_clear=None, **kwargs):
if(self.clear_feature_files_func):
self.clear_feature_files_func(self, features_to_clear=features_to_clear, **kwargs);
else:
VBWARN("CLEAR FEATURE FILES FUNCTION HAS NOT BEEN PROVIDED FOR {} INSTANCE".format(self.VBOBJECT_TYPE()));
########### VIRTUAL FUNCTIONS #############
def AOBJECT_TYPE(self):
return 'VBObject';
def VBOBJECT_TYPE(self):
return self.AOBJECT_TYPE();
# ##Example of how these functions should be written ina subclass, for 'AssetManager' class
# def VBOBJECT_TYPE(self):
# return 'AssetManager';
#
# def toDictionary(self):
# d = VBObject.toDictionary(self);
# #serialize class specific members
# return d;
#
# def initFromDictionary(self, d):
# VBObject.initFromDictionary(self, d);
# #do class specific inits with d;
================================================
FILE: src/video2npz/visbeat3/visbeat3/Video.py
================================================
#VideoVersion#from VisBeatImports import *
from .AObject import *
from .TimeSignal import *
from .Audio import *
from .Warp import *
from .Image import *
import moviepy.editor as mpy
from moviepy.audio.AudioClip import AudioArrayClip as MPYAudioArrayClip
import sys
import math
from operator import truediv
def MPYWriteVideoFile(mpyclip, filename, **kwargs):
temp_audio_filename = get_temp_file_path(final_file_path='TEMP_'+filename+'.m4a', temp_dir_path=Video.VIDEO_TEMP_DIR);
return mpyclip.write_videofile(filename=filename, temp_audiofile= temp_audio_filename, audio_codec='aac', **kwargs);
class Video(TimeSignal):
"""Video (class): A video, and a bunch of convenience functions to go with it.
Attributes:
"""
VIDEO_TEMP_DIR = './'
FEATURE_FUNCS = TimeSignal.FEATURE_FUNCS.copy();
def AOBJECT_TYPE(self):
return 'Video';
def __init__(self, path=None, name=None, num_frames_total=None):
TimeSignal.__init__(self, path=path);
if(name):
self.name = name;
if(path):
self.loadFile(num_frames_total=num_frames_total);
def initializeBlank(self):
TimeSignal.initializeBlank(self);#YES KEEP
self.name = None;
self.sampling_rate = None;
self.audio=None;
self.reader = None;
self.writer = None;
self.num_frames_total=None;
self.reshape=None;
self.meta_data = None;
self.sampling_rate = None;
self.source = None;
# _gui is not saved
self._gui = None;
def _getFrameRate(self):
return self.sampling_rate;
# _gui is from TimeSignal
#
@property
def gui(self):
return self._getGui();
def _getGui(self):
return self._gui;
@gui.setter
def gui(self, value):
self._setGui(value);
def _setGui(self, value):
self._gui = value;
#
def getVersionInfo(self):
"""This is the info to be saved in asset version dictionary"""
d={};
d['name']=self.name;
d['sampling_rate']=self.sampling_rate;
d['num_frames_total']=self.num_frames_total;
d['duration']=self.getDuration();
d['start_time']=self.getStartTime();
d['end_time']=self.getEndTime();
d['meta_data']=self.meta_data;
return d;
def getName(self):
if(self.name is None):
return self.getInfo('file_name');
else:
return self.name;
def getTempDir(self):
if(self.source is not None):
return self.source.getDir('temp');
else:
return Video.VIDEO_TEMP_DIR;
def getStringForHTMLStreamingBase64(self):
svideo = io.open(self.getPath(), 'r+b').read()
encoded = base64.b64encode(svideo)
return "data:video/mp4;base64,{0}".format(encoded.decode('ascii'));
def n_frames(self):
if(not self.num_frames_total):
self.num_frames_total = self.calcNumFramesTotal();
return self.num_frames_total;
def getDuration(self):
return truediv(self.n_frames(), self.sampling_rate);
def getStartTime(self):
return 0;
def getEndTime(self):
return self.getDuration();
def getMPYClip(self, get_audio=True):
return mpy.VideoFileClip(self.getPath(), audio=get_audio);
def getAudio(self):
return self.audio;
def loadFile(self, file_path=None, num_frames_total=None):
if (file_path):
self.setPath(file_path=file_path);
if ('file_path' in self.a_info):
self.reader = imageio.get_reader(self.a_info['file_path'], 'ffmpeg');
self.meta_data = self.reader.get_meta_data();
self.sampling_rate = self.meta_data['fps'];
if (num_frames_total is not None):
self.num_frames_total = num_frames_total;
else:
self.num_frames_total = self.calcNumFramesTotal();
try:
self.audio = Audio(self.a_info['file_path']);
self.audio.name =self.name;
#except RuntimeError:
except Exception:
print(("Issue loading audio for {}".format(self.a_info['file_path'].encode('utf-8'))));
self.audio = Audio(sampling_rate=16000);
self.audio.x = np.zeros(int(np.ceil(self.audio.sampling_rate*self.getDuration())));
def openVideoWriter(self, output_file_path, fps=None):
if('outputs' not in self.a_info):
self.a_info['outputs'] = [];
out_fps = fps;
if(not out_fps):
out_fps=self.sampling_rate;
make_sure_dir_exists(output_file_path);
self.writer = imageio.get_writer(output_file_path, 'ffmpeg', macro_block_size = None, fps = out_fps);
self.a_info['outputs'].append(output_file_path);
def closeVideoWriter(self):
self.writer.close();
self.writer = None;
def getFrameShape(self):
fs=self.getInfo('frame_shape');
if(fs is not None):
return fs;
else:
self.setInfo(label='frame_shape', value=self.reader.get_data(0).shape);
return self.getInfo('frame_shape');
def calcNumFramesTotal(self):
#assert(False)
print(("Calculating frames for {}...".format(self.name)))
valid_frames = 0
example_frame = self.reader.get_data(0);
self.setInfo(label='frame_shape',value=example_frame.shape);
# https://stackoverflow.com/questions/54778001/how-to-to-tackle-overflowerror-cannot-convert-float-infinity-to-integer
#for i in range(1, self.reader.get_length()):
for i in range(1, self.reader.count_frames()):
try:
self.reader.get_data(i);
except imageio.core.format.CannotReadFrameError as e:
break
valid_frames += 1
print("Done.")
return valid_frames
def readFrameBasic(self, i):
"""You should basically never call this"""
fi=i;
if(fi<0):
fi=0;
if(fi>(self.num_frames_total-1)):
fi=(self.num_frames_total-1);
return np.asarray(self.reader.get_data(int(fi)));
def getFrame(self, f):
return self.readFrameBasic(round(f));
def getFrameFromTime(self, t):
f = t*self.sampling_rate;
return self.getFrame(f=f);
def getFrameLinearInterp(self, f):
if(isinstance(f,int) or f.is_integer()):
return self.readFrameBasic(int(f));
prev_frame = self.readFrameBasic(math.floor(f));
next_frame = self.readFrameBasic(math.ceil(f));
frame_progress = f-np.floor(f);
rframe = (next_frame*frame_progress)+(prev_frame*(1.0-frame_progress));
return rframe.astype(prev_frame.dtype);
def writeFrame(self, img):
if self.writer.closed:
print('ERROR: Vid writer object is closed.')
else:
self.writer.append_data(img.astype(np.uint8))
def play(self):
if(ISNOTEBOOK):
print("Playing video:")
video = io.open(self.getPath(), 'r+b').read()
encoded = base64.b64encode(video)
vidhtml=HTML(data=''''''.format(encoded.decode('ascii')));
IPython.display.display(vidhtml);
# return vidhtml;
else:
print("HOW TO PLAY VIDEO? NOT A NOTEBOOK.")
def show(self):
self.play();
def write(self, output_path, output_sampling_rate=None):
assert output_path, "MUST PROVIDE OUTPUT PATH FOR VIDEO"
sampling_rate = output_sampling_rate;
if(not sampling_rate):
sampling_rate=self.sampling_rate;
tempfilepath = get_temp_file_path(final_file_path=output_path, temp_dir_path=self.getTempDir());
self.openVideoWriter(output_file_path=tempfilepath, fps=output_sampling_rate);
duration = self.getDuration();
nsamples = sampling_rate*duration;
old_frame_time = truediv(1.0,self.sampling_rate);
frame_start_times = np.linspace(0,self.getDuration(),num=nsamples,endpoint=False);
frame_index_floats = frame_start_times*self.sampling_rate;
start_timer=time.time();
last_timer=start_timer;
fcounter=0;
for nf in range(len(frame_index_floats)):
self.writeFrame(self.getFrame(frame_index_floats[nf]));
fcounter+=1;
if(not (fcounter%50)):
if((time.time()-last_timer)>10):
last_timer=time.time();
print(("{}%% done after {} seconds...".format(100.0*truediv(fcounter,len(frame_index_floats)), last_timer-start_timer)));
self.closeVideoWriter();
rvid = Video.CreateFromVideoAndAudio(video_path=tempfilepath, audio_object=self.audio, output_path=output_path);
os.remove(tempfilepath);
return rvid;
def VideoClip(self, start=None, end=None, name=None):
from . import VideoClip
if(start is None):
start = 0;
if(end is None):
end = self.getDuration();
clip = VideoClip.VideoClip(video=self, start=start, end=end);
if(name):
clip.name=name;
return clip;
def getImageFromFrame(self, i):
rimage = Image(data=self.getFrame(i));
rimage.setInfo(label='parent_video', value=self.getInfo('file_path'));
rimage.setInfo(label='frame_number',value=i);
return rimage;
def getImageFromTime(self, t):
rimage = Image(data=self.getFrameFromTime(t));
rimage.setInfo(label='parent_video', value=self.getInfo('file_path'));
rimage.setInfo(label='frame_time', value=t);
return rimage;
# def getFrameShape(self):
# return self.getImageFromFrame(0).getShape();
def writeResolutionCopyFFMPEG(self, path, max_height=None):
if(max_height is None):
max_height=self.getFrameShape()[0];
mpc = self.getMPYClip();
clip_resized = mpc.resize(height=max_height); # make the height 360px ( According to moviePy documenation The width is then computed so that the width/height ratio is conserved.)
if((clip_resized.size[0]%2)>0):
clip_resized=clip_resized.crop(x1=0,width=clip_resized.size[0]-1);
# clip_resized.write_videofile(path);
MPYWriteVideoFile(clip_resized, path);
rvid = Video(path);
return rvid;
def writeFFMPEG(self, output_path):
mpc = self.getMPYClip();
# mpc.write_videofile(output_path, preset='fast', codec='mpeg4');
MPYWriteVideoFile(mpc, output_path, preset='fast', codec='mpeg4')
rvid = Video(output_path);
return rvid;
def writeResolutionCopy(self, path, max_height=None, reshape=None, input_sampling_rate_factor = None, output_sampling_rate=None):
print((self.getPath()))
if(input_sampling_rate_factor is not None):
original_sampling_rate = self.sampling_rate;
self.sampling_rate=input_sampling_rate_factor*self.sampling_rate;
original_audio_sampling_rate = self.audio.sampling_rate;
self.audio.sampling_rate=self.audio.sampling_rate*input_sampling_rate_factor;
output_path = path;
inshape = self.getFrameShape();
if(reshape==True):
imshape = [inshape[1],inshape[0],inshape[2]];
outshape = None;
if(max_height and (max_height10):
last_timer=time.time();
print(("{}%% done after {} seconds...".format(100.0*truediv(fcounter,len(frame_index_floats)), last_timer-start_timer)));
self.closeVideoWriter();
rvid = Video.CreateFromVideoAndAudio(video_path=tempfilepath, audio_object=self.audio, output_path=output_path);
os.remove(tempfilepath);
if(input_sampling_rate_factor is not None):
#return original_sampling_rate
self.sampling_rate=original_sampling_rate;
self.audio.sampling_rate = original_audio_sampling_rate;
return rvid;
def writeWarped(self, output_path, warp, output_sampling_rate=None, output_audio=None, bitrate=None, vbmark = True, max_time = None, **kwargs):
sampling_rate = output_sampling_rate;
if (not sampling_rate):
sampling_rate = self.sampling_rate;
duration = self.getDuration();
old_frame_time = truediv(1.0, self.sampling_rate);
target_start = warp.getTargetStart();
target_end = warp.getTargetEnd();
target_duration = target_end - target_start;
if(max_time is not None and target_duration>max_time):
target_duration = max_time;
target_end = target_start+max_time;
print((
"target start: {}\ntarget end: {}\ntarget duration: {}".format(target_start, target_end, target_duration)));
new_n_samples = target_duration * sampling_rate;
target_start_times = np.linspace(target_start, target_end, num=new_n_samples, endpoint=False);
unwarped_target_times = [];
for st in target_start_times:
unwarped_target_times.append(warp.warpTargetTime(st));
frame_index_floats = np.true_divide(np.array(unwarped_target_times), old_frame_time);
tempfilepath = get_temp_file_path(final_file_path=output_path, temp_dir_path=Video.VIDEO_TEMP_DIR);
self.openVideoWriter(output_file_path=tempfilepath, fps=output_sampling_rate, **kwargs);
start_timer = time.time();
last_timer = start_timer;
fcounter = 0;
if(vbmark):
vbmarker = self.getImageFromFrame(0)._vbmarker();
for nf in range(len(frame_index_floats)):
try:
nwfr = self.getFrame(frame_index_floats[nf]);
if(vbmark):
nwfrm = Image(data=nwfr);
nwfrm._splatAtPixCoord(**vbmarker);
nwfr = nwfrm._pixels_uint;
self.writeFrame(nwfr);
except ValueError:
print(("VALUE ERROR ON WRITEFRAME {}".format(frame_index_floats[nf])));
self.writeFrame(self.getFrame(math.floor(frame_index_floats[nf])));
fcounter += 1;
if (not (fcounter % 50)):
if ((time.time() - last_timer) > 10):
last_timer = time.time();
print(("{}%% done after {} seconds...".format(100.0 * truediv(fcounter, len(frame_index_floats)),
last_timer - start_timer)));
self.closeVideoWriter();
silent_warped_vid = Video(tempfilepath);
if (not output_audio):
output_audio = self.getAudio();
cropped_output_audio = output_audio.AudioClip(start=target_start, end=target_end);
if ((self.getInfo('max_height') is None) and (bitrate is None)):
use_bitrate = "20000k";
print(('Using bitrate of {}'.format(use_bitrate)));
rvid = Video.CreateFromVideoAndAudioObjects(video=silent_warped_vid, audio=cropped_output_audio,
output_path=output_path, bitrate=use_bitrate);
elif (bitrate is 'regular'):
rvid = Video.CreateFromVideoAndAudioObjects(video=silent_warped_vid, audio=cropped_output_audio,
output_path=output_path);
else:
rvid = Video.CreateFromVideoAndAudioObjects(video=silent_warped_vid, audio=cropped_output_audio,
output_path=output_path, bitrate=bitrate);
os.remove(tempfilepath);
rvid.setInfo(label='warp_used', value=warp);
return rvid;
def getVersionLabel(self):
return self.getInfo('version_label');
def getWarpsDir(self):
if(self.source is None):
return self.getTempDir();
version_label = self.getVersionLabel()
return self.source.getWarpsDir(version_label=version_label);
def getWithBeginningCroppedToAudio(self, target):
if(isinstance(target, Audio)):
B=target;
else:
B=target.getAudio();
A = self.getAudio();
AB = A.getShiftAmountTo(B);
BA = B.getShiftAmountTo(A);
if(AB n_audio_samples_sig):
nreps = math.ceil(truediv(n_audio_samples_in_vid, n_audio_samples_sig));
if (is_stereo):
audio_sig = np.concatenate(
(audio_sig, np.zeros((2, n_audio_samples_in_vid - n_audio_samples_sig))),
axis=1);
else:
audio_sig = np.tile(audio_sig, (int(nreps)));
audio_sig = audio_sig[:int(n_audio_samples_in_vid)];
if (is_stereo):
# reshapex = np.reshape(audio_sig, (audio_sig.shape[1], audio_sig.shape[0]), order='F');
reshapex = np.transpose(audio_sig);
audio_clip = MPYAudioArrayClip(reshapex, fps=audio_sampling_rate); # from a numeric arra
else:
reshapex = audio_sig.reshape(len(audio_sig), 1);
reshapex = np.concatenate((reshapex, reshapex), axis=1);
audio_clip = MPYAudioArrayClip(reshapex, fps=audio_sampling_rate); # from a numeric arra
video_clip = video_object.getMPYClip();
video_clip = video_clip.set_audio(audio_clip);
# video_clip.write_videofile(output_path,codec='libx264', write_logfile=False);
if (bitrate is None):
# video_clip.write_videofile(output_path, codec=codec, write_logfile=False);
MPYWriteVideoFile(video_clip, output_path, codec=codec, write_logfile=False);
else:
MPYWriteVideoFile(video_clip, output_path, codec=codec, write_logfile=False, bitrate=bitrate);
# video_clip.write_videofile(output_path, codec=codec, write_logfile=False, bitrate=bitrate);
if (return_vid):
return Video(output_path);
else:
return True;
@staticmethod
def CreateByStackingVideos(video_objects=None, video_paths=None, output_path=None, audio = None, concatdim = 0, force_recompute=True, **kwargs):
assert output_path, "MUST PROVIDE OUTPUT PATH FOR VIDEO"
if(not force_recompute):
if(os.path.isfile(output_path)):
return Video(path=output_path);
matchdim = (concatdim+1)%2;
vids=[];
if(video_objects is not None):
vids=video_objects;
if(video_paths is not None):
for vp in video_paths:
vids.append(Video(path=video_paths));
output_path=output_path.encode(sys.getfilesystemencoding()).strip();
make_sure_dir_exists(output_path);
basevid = vids[0];
if(audio is None):
audio = basevid.audio;
sampling_rate = basevid.sampling_rate;
tempfilepath = get_temp_file_path(final_file_path=output_path, temp_dir_path=vids[0].getTempDir());
basevid.openVideoWriter(output_file_path=tempfilepath, fps=sampling_rate);
duration = basevid.getDuration();
nsamples = sampling_rate*duration;
old_frame_time = truediv(1.0,sampling_rate);
#frame_start_times = np.linspace(self.getStartTime(),self.getEndTime(),num=nsamples,endpoint=False);
frame_start_times = np.linspace(0,duration,num=nsamples,endpoint=False);
#frame_index_floats = frame_start_times/old_frame_time;
frame_index_floats = frame_start_times*sampling_rate;
for nf in range(len(frame_index_floats)):
frameind = frame_index_floats[nf];
newframe = basevid.getFrame(frameind);
for vn in range(1,len(vids)):
addpart = vids[vn].getFrame(frameind);
partsize = np.asarray(addpart.shape)[:];
cumulsize = np.asarray(newframe.shape)[:];
if(partsize[matchdim]!=cumulsize[matchdim]):
sz = partsize[:];
sz[matchdim]=cumulsize[matchdim];
addpart = sp.misc.imresize(addpart, size=sz);
newframe = np.concatenate((newframe, addpart), concatdim);
basevid.writeFrame(newframe);
basevid.closeVideoWriter();
rvid = Video.CreateFromVideoAndAudio(video_path=tempfilepath, audio_object=audio, output_path=output_path, **kwargs);
os.remove(tempfilepath);
return rvid;
@staticmethod
def CreateFromVideoAndAudioPaths(video_path, audio_path, output_path, return_vid = True, **kwargs):
return Video.CreateFromVideoAndAudio(video_path=video_path, audio_path=audio_path, output_path=output_path,return_vid=return_vid, **kwargs);
@staticmethod
def CreateFromVideoAndAudioObjects(video, audio, output_path, clip_to_video_length=True, return_vid = True, **kwargs):
return Video.CreateFromVideoAndAudio(video_object=video, audio_object=audio, output_path=output_path, clip_to_video_length=clip_to_video_length, return_vid = return_vid, **kwargs);
def _getDefaultPeakPickingTimeParams(self, **kwargs):
single_frame = np.true_divide(1.0, self._getFrameRate());
time_params = dict(
pre_max_time=2.0 * single_frame,
post_max_time=2.0 * single_frame,
pre_avg_time=5.0 * single_frame,
post_avg_time=5.0 * single_frame,
wait_time=2.0 * single_frame,
delta=0.015,
)
time_params.update(kwargs);
return time_params;
# ###################
# FEATURE
def getFrameIndexes(self, force_recompute=False):
feature_name = 'frame_indexes';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = dict();
duration = self.getDuration();
nsamples = self.sampling_rate * duration;
frame_start_times = np.linspace(0, duration, num=nsamples, endpoint=False);
value = frame_start_times * self.sampling_rate;
self.setFeature(name=feature_name, value=value, params=params);
return self.getFeature(feature_name);
def getFeaturesList(self):
return super(Video, self).getFeaturesList()+self.audio.getFeaturesList();
def getVideoFeaturesList(self):
return super(Video, self).getFeaturesList();
def getFeatureFunctionsList(self):
return super(Video, self).getFeatureFunctionsList()+self.audio.getFeatureFunctionsList();
def getFeatureSourceType(self, name):
"""returns whether video or audio feature, instead of parent version which returns true or false"""
vidhas = super(Video, self).hasFeature(name=name);
if(vidhas):
return 'video';
audiohas = self.audio.hasFeature(name=name);
if(audiohas):
return 'audio';
return None;
def getFeature(self, name, force_recompute=False, **kwargs):
"""Returns None if feature is not already computed, and feature name is not implemented and registered with FEATURE_FUNCS"""
params = kwargs;
assert(not kwargs.get('params')), "STILL TRYING TO USE PARAMS INSTEAD OF KWARGS. FIX THIS";
ftry = super(Video, self).getFeature(name=name, force_recompute=force_recompute, **kwargs);
if(ftry is not None):
return ftry;
else:
return self.audio.getFeature(name=name, force_recompute=force_recompute, **kwargs);
Video.FEATURE_FUNCS['frame_indexes']=Video.getFrameIndexes;
from . import Video_CV;
if(Video_CV.USING_OPENCV):
Video.FEATURE_FUNCS.update(Video_CV.FEATURE_FUNCS);
Video.USING_OPENCV = Video_CV.USING_OPENCV;
Video.localRhythmicSaliencyFunction = Video_CV.localRhythmicSaliencyFunction
Video.visualBeatFunction = Video_CV.visualBeatFunction
Video.cvGetGrayFrame = Video_CV.cvGetGrayFrame;
Video.getImageFromFrameGray = Video_CV.getImageFromFrameGray;
Video.plotVisualBeats = Video_CV.plotVisualBeats;
Video.loadFlowFeatures = Video_CV.loadFlowFeatures;
Video.getVisualBeats = Video_CV.getVisualBeats
Video.getLocalRhythmicSaliency = Video_CV.getLocalRhythmicSaliency;
Video.getDirectogram = Video_CV.getDirectogram;
Video.getDirectogramPowers = Video_CV.getDirectogramPowers;
Video.getVisibleImpactEnvelope = Video_CV.getVisibleImpactEnvelope;
Video.getVisibleImpactEnvelopePowers = Video_CV.getVisibleImpactEnvelopePowers;
Video.getVisibleImpacts = Video_CV.getVisibleImpacts;
Video.getVisualBeats = Video_CV.getVisualBeats;
# Video.getBackwardVisualBeats = Video_CV.getBackwardVisualBeats;
# Video.getForwardVisualBeats = Video_CV.getForwardVisualBeats;
Video.getBackwardVisibleImpactEnvelope = Video_CV.getBackwardVisibleImpactEnvelope;
Video.getBothWayVisibleImpactEnvelope = Video_CV.getBothWayVisibleImpactEnvelope;
Video.getForwardVisibleImpactEnvelope = Video_CV.getForwardVisibleImpactEnvelope;
Video.getDirectionalFlux = Video_CV.getDirectionalFlux;
Video.getVisualTempogram = Video_CV.getVisualTempogram;
Video.getCutEvents = Video_CV.getCutEvents;
Video.computeImpactEnvelope = Video_CV.computeImpactEnvelope;
Video.computeDirectogramPowers = Video_CV.computeDirectogramPowers;
Video.visualBeatsFromEvents = Video_CV.visualBeatsFromEvents;
Video.plotVisualBeats = Video_CV.plotVisualBeats;
Video.plotImpactEnvelope = Video_CV.plotImpactEnvelope;
Video.plotVisibleImpacts = Video_CV.plotVisibleImpacts;
Video.getVisualBeatSequences = Video_CV.getVisualBeatSequences;
Video.printVisualBeatSequences = Video_CV.printVisualBeatSequences;
Video.getVisualBeatTimes = Video_CV.getVisualBeatTimes;
Video.findAccidentalDanceSequences = Video_CV.findAccidentalDanceSequences;
Video.plotEvents = Video_CV.plotEvents;
# Video. = Video_CV.
# Video. = Video_CV.
# Video. = Video_CV.
# Video. = Video_CV.
# Video. = Video_CV.
# Video. = Video_CV.
else:
AWARN("Was not able to add functions that use OpenCV! Check OpenCV instalation and try again?");
from .vbgui import BeatGUI
if(BeatGUI.VIEWER_INSTALLED):
def getEvents(self, **kwargs):
time_params = self._getDefaultPeakPickingTimeParams();
time_params.update(kwargs)
vbeats = self.getFeature('visual_beats', force_recompute=True, **time_params);
return vbeats;
def getEventList(self, **kwargs):
events = self.getEvents(**kwargs);
return EventList(events=events);
def runBeatGUIOnAudio(self):
audio = self.getAudio();
self.gui.run(local_saliency=audio.getLocalRhythmicSaliency(), frame_rate = audio._getFrameRate(), eventlist = audio.getEventList());
def runGUI(self, local_saliency=None, frame_rate = None, eventlist = 'default', frame_offset=None):
self.gui.run(local_saliency=local_saliency, frame_rate=frame_rate, eventlist=eventlist, frame_offset=frame_offset);
Video._getGui = BeatGUI.media_GUI_func;
Video.runGUI = runGUI;
Video.getEvents = getEvents;
Video.getEventList = getEventList;
Video.runBeatGUIOnAudio = runBeatGUIOnAudio;
else:
AWARN("BeatGUI not installed");
================================================
FILE: src/video2npz/visbeat3/visbeat3/VideoClip.py
================================================
from .Video import *
from .AudioClip import *
class VideoClip(Video):
"""VideoClip (class): A segment of a video, and a bunch of convenience functions to go with it.
Attributes:
start: The name of the video
end: The framerate of the video
"""
def AOBJECT_TYPE(self):
return 'VideoClip';
def __init__(self, video=None, start=None, end=None, clip_to_frame=True, path=None):
"""If a video is provided, we don't want to reload the video from disk. If a path is provided, we have no choice.
"""
assert(not (video and path)), "provided both video object and path to VideoClip constructor."
assert((start is not None) and (end is not None)), "must provide start time and end time to AudioClip constructor"
Video.__init__(self, path = path);
self.initializeBlank();
self.start = start;
self.end = end;
if(video):
self.setPath(video.getPath());
self.reader = video.reader;
#self.writer = None; #this only comes up if we write later
self.sampling_rate=video.sampling_rate;
self.num_frames_total=video.num_frames_total;
time_per_frame=truediv(1.0,self.sampling_rate);
if(clip_to_frame):
self.start=time_per_frame*math.floor(truediv(self.start,time_per_frame));
self.end=time_per_frame*math.floor(truediv(self.end,time_per_frame));
if(video.name):
self.name = video.name+"_{}_{}".format(start,end);
self.audio = video.audio.AudioClip(start=start, end=end);
else:
assert(False),"must provide video to VideoClip init"
def initializeBlank(self):
Video.initializeBlank(self);
self.start = None;
self.end = None;
# def readFrameBasic(self, i):
# return Video.getFrame(self, float(i)+self.start);
def getFrameLinearInterp(self, f):
return Video.getFrameLinearInterp(self, float(f)+self.start*self.sampling_rate);
def getFrame(self, f):
return Video.getFrame(self, float(f)+self.start*self.sampling_rate);
def getDuration(self, round_to_frames=False):
if(not round_to_frames):
return self.end-self.start;
else:
return truediv(self.n_frames(), self.sampling_rate);
def n_frames(self):
#return self.getLastFrameIndex()-self.getFirstFrameIndex();
return math.ceil((self.end-self.start)*self.sampling_rate);
# def getFirstFrameIndex(self):
# return math.floor(self.start*self.sampling_rate);
#
# def getLastFrameIndex(self):
# #I think floor is still right here, because the times mark beginnings of frames
# return math.floor(self.end*self.sampling_rate);
def getStartTime(self):
return self.start;
def getEndTime(self):
return self.end;
def getMPYClip(self, get_audio=True):
return mpy.VideoFileClip(self.getPath(), audio=get_audio).subclip(self.getStartTime(), self.getEndTime());
def play(self):
if(ISNOTEBOOK):
print("Playing video:")
IPython.display.display(self.getMPYClip().ipython_display(fps=self.sampling_rate, maxduration=self.getDuration()+1));
else:
print("HOW TO PLAY VIDEO? NOT A NOTEBOOK.")
================================================
FILE: src/video2npz/visbeat3/visbeat3/VideoSource.py
================================================
from .VisBeatImports import *
from .Video import *
from .AFileManager import AFileManager
try:
import youtube_dl
except ImportError:
AWARN('You need to install youtube-dl to use parts of VideoSource class that pull from YouTube. Try running:\npip install youtube-dl')
def safe_file_name(input_string):
return ''.join([i if ord(i) < 128 else '_' for i in input_string]);
class VideoSource(AFileManager):
"""Video (class): A video, and a bunch of convenience functions to go with it.
Attributes:
name: The name of the video
sampling_rate: The framerate of the video
audio: an Audio object, the audio for the video
"""
VIDEO_TEMP_DIR = './'
def getJSONName(self):
return self.AOBJECT_TYPE()+".json";
def AOBJECT_TYPE(self):
return 'VideoSource';
def __init__(self, path, name=None, source_location=None, VideoClass = None, **kwargs):
"""
:param path: either the path to a json, or the path to a directory containing 'VideoSource.json',
or the path to a directory where said json should be created.
:param name: name for video
:param source_location: can be a path to a video file, or a youtube source from which to pull a video file
:param VideoClass: visbeat.Video by default, but can be changed if you are using a custom subclass.
Must be a subclass of visbeat.Video, and must be constructable with VideoClass(path)
"""
AFileManager.__init__(self, path=path);
if(VideoClass is not None):
self.VideoClass = VideoClass;
self.setSource(source_location=source_location, assert_valid=None, **kwargs);
# if (self.name is None):
# self.name = os.path.splitext(os.path.basename(path))[0]
@staticmethod
def NewVideoSource(destination, name, source_location=None, VideoClass = None, **kwargs):
vsdir = os.path.join(destination, name+os.sep);
make_sure_dir_exists(vsdir);
print(("Video source at {}".format(vsdir)));
return VideoSource(path=vsdir, name=name, source_location=source_location, **kwargs);
def initializeBlank(self):
AFileManager.initializeBlank(self);
self.source_location = None;
self.name = None;
self.video_file_name = None;
self.title_safe = None;
self.youtube_info_dict = None;
self.versions_info = {};
#
self.VideoClass = Video;
def toDictionary(self):
d = AFileManager.toDictionary(self);
d['source_location']=self.source_location;
if(self.name):
d['name']=self.name;
if(self.video_file_name):
d['video_file_name']=self.video_file_name;
if(self.title_safe):
d['title_safe']=self.title_safe;
if(self.youtube_info_dict):
d['youtube_info_dict']=self.youtube_info_dict;
if(self.versions_info):
d['versions_info']=self.versions_info;
return d;
def initFromDictionary(self, d):
AFileManager.initFromDictionary(self, d);
self.source_location = d['source_location'];
self.name = d.get('name');
self.video_file_name = d.get('video_file_name');
self.title_safe = d.get('title_safe');
self.youtube_info_dict = d.get('youtube_info_dict');
self.versions_info=d.get('versions_info');
if(self.versions_info is None):
self.versions_info = {};
#do class specific inits with d;
def initWithPath(self, path=None, clear_temp=None):
AFileManager.initWithPath(self, path=path, clear_temp=clear_temp);
self.setDir('versions', pathstring(self.getDirectoryPath() + os.sep + "Versions" + os.sep));
self.setDir('warps', pathstring(self.getDir('versions')+ os.sep + "Warps" + os.sep));
# self.setDir('midi', pathstring(self.getDirectoryPath() + os.sep + "MIDI" + os.sep));
# self.setDir('music', pathstring(self.getDirectoryPath() + os.sep + "Music" + os.sep));
self.setFeaturesDir();
def setFeaturesDir(self, features_dir=None):
if (features_dir is None):
self.setDir('features', pathstring(self.getDir('data') + os.sep + "Features" + os.sep))
else:
self.setDir('features', features_dir);
def getName(self):
"""
Gets name if set. If not set, returns file name without extension.
:return:
"""
if(self.name is not None):
return self.name;
elif(self.video_file_name is not None):
return os.path.splitext(self.video_file_name)[0];
else:
return None;
###################
@staticmethod
def _versionLabelString(version_label=None):
if (version_label and version_label != 'Full'):
return str(version_label);
else:
return 'Full';
@staticmethod
def _versionGroupString(version_group=None):
if (version_group is None):
version_group = 'Original';
return version_group;
@staticmethod
def _getVersionLabelDirName(version_label=None):
if(isinstance(version_label, int)):
return ("maxheight_{}".format(version_label));
else:
return "Full";
def getVersionPath(self, version_label=None, version_group=None):
"""
Get path to version of video (path to video file). Return None if version has not been added.
:param version_label:
:return:
"""
return self.getVersionInfo(version_label=version_label, version_group=version_group, info_label='path');
def getDirForVersion(self, version_label=None, version_group=None):
vdir = self.getDir('versions') + os.sep + self._versionGroupString(version_group) + os.sep + self._getVersionLabelDirName(version_label)+os.sep;
return vdir;
def getVersionInfo(self, version_label, version_group=None, info_label=None):
"""
Get info about a version of the video
:param version_label:
:param info_label:
:return:
"""
assert(info_label is not None), "should provide info_label to getVersionInfo()"
d = self.getVersionDictionary(version_label=version_label, version_group=version_group);
if(d is not None):
return d.get(info_label);
else:
return None;
def getVersionDictionary(self, version_label=None, version_group=None):
if(version_group is None):
version_group='Original';
vis_d=self.versions_info.get(version_group);
if(vis_d is not None):
return vis_d.get(VideoSource._versionLabelString(version_label));
else:
return None;
def setVersionDictionary(self, version_label=None, version_group=None, d=None):
if(version_group is None):
version_group='Original';
vis_d=self.versions_info.get(version_group);
if(vis_d is None):
self.versions_info[version_group]={};
vis_d=self.versions_info.get(version_group);
vis_d[VideoSource._versionLabelString(version_label)]=d;
return;
def setVersionInfo(self, version_label=None, version_group=None, video_path=None, **kwargs):
version_dict = self.getVersionDictionary(version_label=version_label, version_group=version_group);
if(version_dict is None):
self.setVersionDictionary(version_label=version_label, version_group=version_group, d={});
version_dict = self.getVersionDictionary(version_label=version_label, version_group=version_group);
if(video_path is not None):
version_dict.update({'path':str(pathstring(video_path))});
if(kwargs is not None):
version_dict.update(kwargs);
####################
def hardSave(self):
if (os.path.isfile(self.getJSONPath())):
os.rename(self.getJSONPath(), self.getDir('backup') + os.sep + self.AOBJECT_TYPE() + ".json");
self.writeToJSON(self.getJSONPath());
def save(self):
if (self.getInfo('block_writing_to_json')):
return;
self.hardSave();
####################
# def removeAllVersionFiles(self, asset_manager=None):
# AWARN("This should wipe the directory, ya?")
# # AWARN("Remove all version files does not remove features and vis images! This still needs to be implemented!")
# # for vtype in self.versions_info:
# # for v in self.versions_info.get(vtype):
# # # print(self.versions_info.get(vtype).get(v));
# # vpath = self.versions_info.get(vtype).get(v).get('path');
# # assert(vpath), "version {} did not have path".format(vtype)
# # print(vpath)
# # if(os.path.isfile(vpath)):
# # print("REMOVING {}".format(vpath))
# # os.remove(vpath);
# # if(asset_manager is not None and vtype=='Original'):
# # if(v=='Full'):
# # rs=None;
# # else:
# # rs = int(v);
# # asset_manager.removeFeaturesForLinkRes(link=self, max_height=rs);
# # return True;
# def removeFeaturesForRes(self, max_height=None):
# v = self.getVersionVideo(name=link.name, max_height = max_height);
# v.clearFeatureFiles(features_to_clear=['all', 'each']);
#
#
# def removeFeatureFilesForVideo(self, video, features_to_remove=None):
# if(features_to_remove is None):
# return;
# if(not isinstance(features_to_remove, list)):
# features_to_remove=[features_to_remove];
# for f in features_to_remove:
# self.removeFeatureFileForVideo(video=video, feature_name=f);
#
#
# def removeFeatureFileForVideo(self, video, feature_name):
# AWARN("still need to implement removeFeatureFileForVideo. Should just empty corresponding directory. Also implement 'each' that removes all.")
# # if(feature_name=='each'):
# # return self.removeFeatureFilesForVideo(video=video, features_to_remove=video.getFeatureFunctionsList());
# # max_height = video.getInfo(label='max_height');
# # link = self.hasLinkWithName(name=video.name);
# # source_type=video.getFeatureSohahurceType(feature_name);
# # ipath = self.getFeaturePathForLink(feature_name=feature_name, link=link, max_height=max_height, source_type=source_type);
# # if(os.path.isfile(ipath)):
# # print("REMOVING {}".format(ipath));
# # os.remove(ipath);
#
# def removeFeatureFiles(self, video, feature_name):
# if(feature_name=='each'):
# AWARN("IMPLEMENT DELETE FEATURE DIR AND CREATE CLEAN");
# version_label = video.getInfo(label='version_label');
# AWARN("IMPLEMENT DELETE {} FOR VERSION_LABEL {}".format(feature_name, version_label));
# # ipath = self.getFeaturePathForLink(feature_name=feature_name, link=link, max_height=max_height, source_type=source_type);
# # if(os.path.isfile(ipath)):
# # print("REMOVING {}".format(ipath));
# # os.remove(ipath);
#
# def removeVersion(self, version_label=None, remove_files=True):
# AWARN("Remove Resolution does not remove features! This still needs to be implemented!")
# for vtype in self.versions_info:
# v = self.versions_info.get(vtype).get(VideoSource._versionLabelString(version_label));
# if(v is not None):
# vpath = v.get('path');
# if(os.path.isfile(vpath)):
# print("REMOVING {}".format(vpath))
# os.remove(vpath);
# self.versions_info.get(vtype).pop(VideoSource._versionLabelString(version_label), None);
# return True;
#
# def copyResolutionTo(self, version_label=None, output_dir=None):
# assert(output_dir)
# for vtype in self.versions_info:
# v = self.versions_info.get(vtype).get(VideoSource._versionLabelString(version_label));
# if(v is not None):
# vpath = v.get('path');
# if(os.path.isfile(vpath)):
# opath = os.path.join(output_dir,(vtype+'_'+VideoSource._versionLabelString(version_label)+self.video_file_name));
# print("COPYING {} to {}".format(vpath,opath));
# shutil.copy2(vpath, opath);
# return True;
#
# def copyAssetsTo(self, output_dir=None, asset_manager=None):
# for vtype in self.versions_info:
# output_type_dir=pathstring(output_dir+os.sep+vtype+os.sep);
# make_sure_dir_exists(output_type_dir);
# for v in self.versions_info.get(vtype):
# output_a_dir = pathstring(output_type_dir+os.sep+v+os.sep);
# make_sure_dir_exists(output_a_dir);
# # print(self.versions_info.get(vtype).get(v));
# vpath = self.versions_info.get(vtype).get(v).get('path');
# if(vpath and os.path.isfile(vpath)):
# print("copying {} to {}".format(vpath, output_a_dir));
# shutil.copy2(vpath, output_a_dir);
# if(asset_manager is not None and vtype=='Original'):
# if(v=='Full'):
# rs=None;
# else:
# rs = int(v);
# asset_manager.saveFeaturesForVersion(version_label=rs, output_dir = output_a_dir);
def getWarpsDir(self, version_label=None):
warpdir = self.getDir('warps');
resdir = self._getVersionLabelDirName(version_label=version_label);
rdir = pathstring(warpdir + os.sep + resdir + os.sep);
make_sure_path_exists(rdir);
return rdir;
# ############################# FROM ASSETMANAGER #######################
def getVersion(self, max_height=None, get_if_missing=True, load_features=True, **kwargs):
"""
Gets a version of the video with maximum height max_height. This function contains logic to decide whether/when
to pull the video. Use pullVersion to force pulling that overwrite's existing assets.
:param max_height:
:param get_if_missing:
:return:
"""
vpath = self.getVersionPath(version_label=max_height);
if(vpath and os.path.isfile(vpath)):
num_frames_total = self.getVersionInfo(version_label=max_height, info_label='num_frames_total');
if(num_frames_total):
v = self.RegisteredVideo(path=vpath, version_label = max_height, num_frames_total=num_frames_total, load_features=True);
else:
v = self.RegisteredVideo(path=vpath, version_label = max_height, load_features=True);
self.setVersionInfo(version_label=max_height, num_frames_total=v.num_frames_total);
self.save();
return v;
else:
if(get_if_missing):
vpath = self.pullVersion(max_height=max_height, **kwargs);
if(vpath):
num_frames_total = self.getVersionInfo(version_label=max_height, info_label='num_frames_total');
if(num_frames_total):
v = self.RegisteredVideo(path=vpath, version_label = max_height, num_frames_total=num_frames_total, load_features=True);
else:
v = self.RegisteredVideo(path=vpath, version_label=max_height, load_features=True);
self.setVersionInfo(version_label=max_height, num_frames_total=v.num_frames_total);
self.save();
else:
AWARN("COULDNT GET VIDEO FROM {}".format(self.source_location));
else:
AWARN("COULDNT FIND VIDEO LOCALLY");
return;
return v;
def saveFeaturesForVideo(self, video, features_to_save=None, output_dir=None, overwrite=True):
if(features_to_save is None):
return;
if(not isinstance(features_to_save, list)):
features_to_save=[features_to_save];
for f in features_to_save:
self.saveFeatureForVideo(video=video, feature_name=f, output_dir=output_dir, overwrite=overwrite);
def saveFeatureForVideo(self, video, feature_name, output_dir=None, overwrite=True):
if(feature_name=='each'):
return self.saveFeaturesForVideo(video=video, features_to_save=video.getFeaturesList(), output_dir=output_dir, overwrite=overwrite);
version_label = video.getInfo(label='version_label');
source_type = video.getFeatureSourceType(feature_name);
opath = self.getFeaturePath(feature_name=feature_name, version_label=version_label, source_type=source_type, output_dir=output_dir);
make_sure_dir_exists(opath);
if(not os.path.isfile(opath) or overwrite):
if(feature_name=='all'):
video.saveFeatures(path=opath);
return;
else:
vfeature = video.getFeature(name=feature_name, force_recompute=False);
if(vfeature is not None):
video.saveFeature(name=feature_name, path=opath);
def loadFeaturesForVideo(self, video, features_to_load=None):
if(features_to_load is None):
return;
if(not isinstance(features_to_load, list)):
features_to_load=[features_to_load];
for f in features_to_load:
self.loadFeatureForVideo(video=video, feature_name=f);
def loadFeatureForVideo(self, video, feature_name):
if(feature_name=='each'):
return self.loadFeaturesForVideo(video=video, features_to_load=video.getFeatureFunctionsList());
version_label = video.getInfo(label='version_label');
source_type=video.getFeatureSourceType(feature_name);
ipath = self.getFeaturePath(feature_name=feature_name, version_label=version_label, source_type=source_type);
if(os.path.isfile(ipath)):
if(feature_name=='all'):
video.loadFeatures(path=ipath);
else:
video.loadFeature(name=feature_name, path=ipath);
def getFeaturePath(self, feature_name=None, source_type=None, version_label=None, output_dir=None):
assert(feature_name is not None), 'must provide name of feature VideoSource.getFeaturePath'
feature_dir = self.getFeatureDir(feature_name=feature_name, source_type = source_type);
fileext = '.pkl';
if(output_dir is None):
version = self._getVersionLabelDirName(version_label=version_label)+os.sep;
output_dir = pathstring(feature_dir);
outname = self.getName();
if(outname is None):
outname = 'version';
outname = outname+'_'+self._getVersionLabelDirName(version_label=version_label)+fileext;
opath = os.path.join(output_dir, outname);
return opath;
def getFeatureDir(self, feature_name=None, source_type=None):
if(source_type is None):
source_type='video'; # could be 'audio'
# AWARN("getDir('features')= {}\nsource_type= {}\nfeature_name= {}".format(self.getDir('features'), source_type, feature_name))
ftypedir = pathstring(self.getDir('features') + os.sep + source_type + os.sep + feature_name + os.sep);
return ftypedir;
def saveFeaturesForVersion(self, version_label=None, output_dir=None):
assert(output_dir), 'must provide output dir';
v = self.getVersion(max_height=version_label);
v.save(features_to_save='all', output_dir=output_dir);
def RegisteredVideo(self, path=None, version_label=None, version_group = None, num_frames_total=None, load_features = True):
v = self.VideoClass(path=path, name=self.getName()+'_'+self._versionLabelString(version_label=version_label), num_frames_total=num_frames_total);
# v = Video(path=path, name=self.getName()+'_'+self._versionLabelString(version_label=version_label), num_frames_total=num_frames_total);
self.RegisterVideo(video=v, version_label=version_label, load_features=load_features);
return v;
def RegisterVideo(self, video, version_label = None, version_group = None, load_features=True):
def videosave(vidob, features_to_save='all', output_dir=None, overwrite=True):
self.saveFeaturesForVideo(video=vidob, features_to_save=features_to_save, output_dir=output_dir, overwrite=overwrite);
#self.save
def videoload(vidob, features_to_load='all', input_dir=None):
self.loadFeaturesForVideo(video=vidob, features_to_load=features_to_load);
def videoclear(vidob, features_to_clear='all'):
self.removeFeatureFilesForVideo(video=vidob, features_to_remove=features_to_clear);
video.setInfo(label='version_label', value=version_label);
video.setInfo(label='version_group', value=version_group);
video.setInfo(label='video_file_name', value=os.path.split(video.getPath())[1]);
# if(version_group is None):
video.save_func = videosave;
video.load_func = videoload;
video.clear_feature_files_func = videoclear;
video.source=self;
if(load_features is True and hasattr(video, "loadFlowFeatures")):
video.loadFlowFeatures();
elif(load_features is not None and (isinstance(load_features, list) or isinstance(load_features, str))):
video.load(features_to_load = load_features);
# return video;
def addVersion(self, path, version_label=None, version_group=None):
v = self.RegisteredVideo(path=path, version_label=version_label);
video_dict=v.getVersionInfo();
video_dict.update(dict(version_group=version_group));
self.setVersionInfo(video_path = path, version_label=version_label, **video_dict);
self.save();
def addVersionToVideo(self, video, new_video, version_label=None, version_group=None):
version_label = video.getInfo(label='version_label');
video_dict = new_video.getVersionInfo();
if ((version_group is not None) or (video_dict.get('version_group') is None)):
video_dict.update(dict(version_group=version_group));
self.setVersionInfo(video_path=new_video.getPath(), version_label=version_label, **video_dict);
self.save();
def setSource(self, source_location=None, assert_valid=True, **kwargs):
"""
Sets the source of this VideoSource. If file, copies the file to VideoSource directory as "Original" version,
unless copy argument is set to false.
:param source_location: either path to file, or youtube url
:param kwargs: see setSourceYoutube or setSourceFile for options
:return:
"""
if(source_location):
if(os.path.isfile(source_location)):
self.setSourceFile(path = source_location, **kwargs);
else:
self.setSourceYoutube(url=source_location, **kwargs);
return;
elif(kwargs.get('video_path')):
self.setSourceFile(**kwargs);
return;
elif(kwargs.get('url')):
self.setSourceYoutube(**kwargs);
return;
else:
if(assert_valid):
assert(False),'No valid source location provided to setSource.'
def setSourceYoutube(self, url=None, max_height=None, pull_fullres=True, **kwargs):
self.source_location=url;
self.setInfo(label='source_type', value='youtube');
if(pull_fullres):
self.pullYoutubeVideo(max_height=max_height, **kwargs);
self.save();
def setSourceFile(self, path=None, copy=True, **kwargs):
# assert(os.path.isfile(path)), 'Tried to set source but no file exists at {}'.format(path);
if(not os.path.isfile(path)):
return None;
self.video_file_name = os.path.basename(path);
if(not copy):
self.source_location = path;
self.setInfo(label='source_type', value='file_address');
return path;
# output_dir = self.getDir('versions') + os.sep + 'Original' + os.sep;
output_dir = self.getDirForVersion(version_label=None, version_group='Source');
make_sure_dir_exists(output_dir);
output_path = os.path.join(output_dir, self.video_file_name);
# max_height = kwargs.get('max_height');
# if(max_height is not None):
# # print('max_height was {}'.format(max_height));
# original = self.VideoClass(path=path);
# original.writeResolutionCopyFFMPEG(path=output_path, max_height=kwargs.get('max_height'));
# else:
# shutil.copy2(path, output_path);
shutil.copy2(path, output_path);
# self.addVersion(path=output_path, version_label='Original');
self.addVersion(path=output_path, version_label='Full');
self.setInfo(label='source_type', value='file');
self.source_location = output_path;
self.save();
return output_path;
def pullVersion(self, max_height=None, **kwargs):
# assert(self.source_location is not None), "Could not pull version; source location is None."
if(self.source_location is None):
return None;
source_type = self.getInfo('source_type');
if(source_type=='youtube'):
return self.pullYoutubeVideo(max_height=max_height, **kwargs);
else:
return self.pullFileVideo(max_height=max_height, **kwargs)
def pullFileVideo(self, max_height=None, force_recompute=None):
original_path = self.source_location;
assert(os.path.isfile(original_path)), 'SOURCE FILE {} IS MISSING'.format(self.source_location);
original = self.VideoClass(path=original_path);
output_dir = self.getDirForVersion(version_label=max_height, version_group=None);
make_sure_dir_exists(output_dir);
output_path = os.path.join(output_dir, self.video_file_name);
if(max_height is None):
if(os.path.normpath(original_path) != os.path.normpath(output_path)):
shutil.copy2(original_path, output_path);
else:
original.writeResolutionCopyFFMPEG(path=output_path, max_height=max_height);
self.addVersion(path=output_path, version_label=max_height);
return self.getVersionPath(version_label=max_height);
def pullYoutubeVideo(self, max_height=None, write_subtitles=False, save_youtube_info=False,
force_redownload=False):
"""
Downloads video from youtube with height<=max_height. Returns path to downloaded video.
:param max_height: maximum height of video to download
:param write_subtitles: whether to save the subtitles
:param save_youtube_info: whether to save the info json
:param force_redownload: whether to re-download if video exists
:return:
"""
assert (not self.getInfo('source_is_file')), "tried to call pullYoutubeVideo on file source {}.".format(self.getName());
old_vid_path = self.getVersionPath(version_label=max_height);
if ((not force_redownload) and old_vid_path and os.path.isfile(old_vid_path)):
AWARN("Old video version exists with path {}\nSkipping download...\n".format(old_vid_path))
return old_vid_path;
# output_dir = self.getDir('versions') + os.sep + self._getVersionLabelDirName(version_label=max_height) + os.sep;
output_dir = self.getDirForVersion(version_label=max_height, version_group=None);
make_sure_path_exists(output_dir);
if ((not force_redownload) and self.video_file_name is not None):
path_guess = os.path.join(output_dir, self.video_file_name);
if (os.path.isfile(path_guess)):
AWARN("found existing file {}\nSkipping download; set force_redownload=True to overwrite old version.".format(path_guess));
self.addVersion(path=path_guess, version_label=max_height);
return path_guess;
########Download From YouTube#########
vidpath_template = output_dir + '%(title)s-%(id)s' + '.%(ext)s';
ydl_opts = {
'format': 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best',
'outtmpl': vidpath_template,
}
if (write_subtitles):
ydl_opts['writesubtitles'] = True;
ydl_opts['subtitlesformat'] = '[srt]';
if (max_height):
ydl_opts['format'] = 'bestvideo[height<={}][ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best'.format(
max_height);
ydl_opts['writeinfojson'] = True;
info_dict = None;
AWARN("Downloading {} from {}...".format(self.getName(), self.source_location));
with youtube_dl.YoutubeDL(ydl_opts) as ydl:
info_dict = ydl.extract_info(self.source_location, download=True)
# video_source = info_dict.get("source", None)
# video_id = info_dict.get("id", None)
# video_title = info_dict.get('title', None)
usedtitle = info_dict['title'].replace('"', "'").replace('|', '_').replace(':', ' -').replace('/','_').replace('?', '');
usedfilename_withoutext = usedtitle + '-' + info_dict['id'];
usedfilename = usedfilename_withoutext + '.' + info_dict['ext'];
filepath = output_dir + os.sep + usedfilename;
filepathsafe = safe_file_name(filepath);
if (os.path.isfile(filepath)):
os.rename(filepath, filepathsafe);
else:
fpath = glob.glob(output_dir + os.sep + '*' + info_dict['id'] + '.mp4');
assert (len(fpath) == 1), "Wrong number of files for {}\nFound:\n{}".format(filepath, fpath);
os.rename(fpath[0], filepathsafe);
jpath = glob.glob(output_dir + os.sep + '*' + info_dict['id'] + '.info.json');
os.rename(jpath[0], safe_file_name(usedfilename_withoutext + '.info.json'));
print(("Saved to {}".format(filepathsafe)));
self.video_file_name = safe_file_name(usedfilename);
self.title_safe = safe_file_name(usedtitle);
if (save_youtube_info):
self.youtube_info_dict = info_dict;
# v = Video(path=filepathsafe);
# video_dict=v.toDictionary();
# self.setVersionInfo(video_path = filepathsafe, version_label=max_height, num_frames_total=v.num_frames_total, **video_dict);
# self.save();
self.addVersion(path=filepathsafe, version_label=max_height);
return self.getVersionPath(version_label=max_height);
# return youtube_link;
================================================
FILE: src/video2npz/visbeat3/visbeat3/Video_CV.py
================================================
# from Video import *
from .Image import *
from .Event import *
from .VisualBeat import *
import librosa
FEATURE_FUNCS = {};
VIS_FUNCS = {};
VISVIDEO_FUNCS = {};
FLOW_LOG_EPSILON=1.0;
FLOW_UNIT_GAIN=10000.0;
HISTOGRAM_FRAMES_PER_BEAT = 2;
HISTOGRAM_DOWNSAMPLE_LEVELS = 3;
VB_UPSAMPLE_FACTOR = 1.0;
USING_OPENCV = Image.USING_OPENCV;
if(USING_OPENCV):
##########################################################################
# ################ THESE ARE THE FUNCTIONS TO OVERRIDE! ################ #
# ################ FOR CUSTOM SALIENCY METRICS! ################ #
##########################################################################
# You can modify them here. But I would suggest elsewhere in your code
# just setting Video.localRhythmicSaliencyFunction and
# Video.visualBeatFunction to whatever you want. See how they are assigned
# when Video_CV is included in Video.py. -Abe
def localRhythmicSaliencyFunction(self, **kwargs):
"""
Change to use different function for local saliency
"""
return self.getVisibleImpactEnvelope(**kwargs);
def visualBeatFunction(self, **kwargs):
"""
Change to use different default strategy for selecting visual beats
"""
# beat_params = dict(
# pre_max_time=0.2,
# post_max_time=0.2,
# pre_avg_time=0.2,
# post_avg_time=0.2,
# wait_time=0.1,
# )
beat_params = self._getDefaultPeakPickingTimeParams();
beat_params.update(kwargs);
return self.getVisibleImpacts(**beat_params);
# #################### This is how they are called #################### #
def getLocalRhythmicSaliency(self, force_recompute=False, **kwargs):
feature_name = 'local_rhythmic_saliency';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = dict();
params.update(kwargs);
params.update({'force_recompute':force_recompute});
result = self.localRhythmicSaliencyFunction(**params);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getVisualBeats(self, force_recompute=False, **kwargs):
feature_name = 'visual_beats';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
params.update({'force_recompute': force_recompute});
result = self.visualBeatFunction(**params);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
# ##################################################################### #
##########################################################################
# ###################################################################### #
##########################################################################
def cvGetGrayFrame(self, f):
colorframe = self.getFrame(f).astype(np.uint8);
return ocv.cvtColor(colorframe, ocv.COLOR_RGB2GRAY);
def getImageFromFrameGray(self, f):
frame = self.getImageFromFrame(f);
frame.RGB2Gray();
return frame;
def flow2row(ang, amp, bins, subdivs, n_shifts, density):
h, w = ang.shape[:2];
ncells = np.power(4, subdivs);
nperd = np.power(2, subdivs);
xw = w-n_shifts[1];
yw = h-n_shifts[0];
xcells = np.arange(nperd + 1, dtype=float);
ycells = np.arange(nperd + 1, dtype=float);
xcells = np.floor((xcells / (nperd)) * xw);
ycells = np.floor((ycells / (nperd)) * yw);
# print(n_shifts)
ahis = np.zeros([ncells * bins, n_shifts[0]*n_shifts[1]]);
for dy in range(n_shifts[0]):
for dx in range(n_shifts[1]):
ampwin = amp[dy:dy+yw,dx:dx+xw];
angwin = ang[dy:dy+yw,dx:dx+xw];
cell_counter = 0;
for x in range(nperd):
for y in range(nperd):
ystart=int(ycells[y]);
yend = int(ycells[y+1]);
xstart = int(xcells[x]);
xend = int(xcells[x + 1]);
angcell = angwin[ystart:yend, xstart:xend];
ampcell = ampwin[ystart:yend, xstart:xend];
cahis, cbinbounds = np.histogram(angcell.ravel(), bins=bins, range=(0, 2 * np.pi),
weights=ampcell.ravel(), density=density);
# print("ahis shape: {}\ncahis shape: {}\ndx, dy: {}, {}\ncell_counter: {}\nbins: {}\n".format(ahis.shape, cahis.shape, dx, dy, cell_counter, bins));
ahis[cell_counter * bins:(cell_counter + 1) * bins, dx+dy*n_shifts[0]] = cahis;
cell_counter = cell_counter + 1;
return ahis;
def getFlowFrame(self, frame_index):
prev_frame = self.cvGetGrayFrame(frame_index-1);
this_frame = self.vbGetGrayFrame(frame_index);
return cvDenseFlowFarneback(from_image=prev_frame, to_image=this_frame);
def getFlowFramePolar(self, frame_index):
"""
:param self:
:param frame_index:
:return: polar where polar[:,:,0] is amplitude, and polar[:,:,1] is angle
"""
flow = self.getFlowFrame(frame_index);
fx, fy = flow[:,:,0], flow[:,:,1];
polar = np.zeros(size(flow));
polar[:,:,0] = np.sqrt(fx * fx + fy * fy);
polar[:,:,1] = np.arctan2(fy, fx) + np.pi
return polar;
def computeDirectogramPowers(self, bins=None, dead_zone= 0.05, density=None, save_if_computed=True, noise_floor_percentile=None, **kwargs):
if(bins is None):
bins = 128;
if(noise_floor_percentile is None):
noise_floor_percentile = 20;
print(("Computing Flow Features with deadzone {}".format(dead_zone)))
signal_dim = 128;
m_histvals = np.zeros([signal_dim, self.n_frames(), 3]);
flow_averages = np.zeros([self.n_frames(), 1]);
sampling_rate=self.sampling_rate;
duration = self.getDuration();
nsamples = sampling_rate*duration;
# print(sampling_rate, duration)
frame_start_times = np.linspace(0,duration,num=int(nsamples),endpoint=False);
frame_index_floats = frame_start_times*self.sampling_rate;
lastframe = self.cvGetGrayFrame(frame_index_floats[0]);
start_timer=time.time();
last_timer=start_timer;
fcounter=0;
counter = 0;
for nf in range(len(frame_index_floats)):
nextframe= self.cvGetGrayFrame(frame_index_floats[nf]);
flow = cvDenseFlowFarneback(from_image=lastframe, to_image=nextframe);
h, w = flow.shape[:2];
fx, fy = flow[:,:,0], flow[:,:,1];
# if(filter_median):
# fx = fx-np.median(fx.ravel());
# fy = fy-np.median(fy.ravel());
# assert(False), "SHOULDNT BE FILTERING MEDIAN! VESTIGIAL CODE"
ang = np.arctan2(fy, fx) + np.pi
amp = np.sqrt(fx*fx+fy*fy);
winstarty = int(dead_zone*h);
winendy = h-winstarty;
winstartx = int(dead_zone*w);
winendx = w-winstartx;
angw = ang[winstarty:winendy, winstartx:winendx];
ampw = amp[winstarty:winendy, winstartx:winendx];
mask0 = (ampw>np.percentile(ampw,noise_floor_percentile)).astype(float);
ahis0, cbinbounds = np.histogram(angw.ravel(), bins=bins, range=(0, 2 * np.pi),
weights=mask0.ravel(), density=density);
ahis1, cbinbounds1 = np.histogram(angw.ravel(), bins=bins, range=(0, 2 * np.pi),
weights=ampw.ravel(), density=density);
ahis2, cbinbounds2 = np.histogram(angw.ravel(), bins=bins, range=(0, 2 * np.pi),
weights=np.power(ampw, 2).ravel(), density=density);
m_histvals[:, counter, 0] = ahis0;
m_histvals[:, counter, 1] = ahis1;
m_histvals[:, counter, 2] = ahis2;
lastframe=nextframe;
counter+=1;
fcounter+=1;
if(not (fcounter%50)):
if((time.time()-last_timer)>10):
last_timer=time.time();
print(("{}%% done after {} seconds...".format(100.0*truediv(fcounter,len(frame_index_floats)), last_timer-start_timer)));
params = dict( bins = bins,
deadzone=dead_zone,
density=density);
params.update(kwargs);
self.setFeature(name='directogram_powers', value=m_histvals, params=params);
if(save_if_computed):
self.save(features_to_save=['directogram_powers']);
return m_histvals;
######################### Other Features #############################
def getDirectogramPowers(self, force_recompute=False, **kwargs):
feature_name = 'directogram_powers';
if ((not self.hasFeature(feature_name)) or force_recompute):
flow_powers = self.computeDirectogramPowers(**kwargs);
return self.getFeature(feature_name);
# def getDirectogram(self, bins = None, weights=None, density=None, force_recompute=False, save_if_computed=True, **kwargs):
def getDirectogram(self, **kwargs):
feature_name = 'directogram';
force_recompute = kwargs.get('force_recompute');
if((not self.hasFeature(feature_name)) or force_recompute):
flow_powers = self.getFeature('directogram_powers');
fh = flow_powers[:,:,1];
self.setFeature(name='directogram', value=fh, params=kwargs);
return self.getFeature(feature_name);
def getVisualTempo(self, force_recompute=None, **kwargs):
feature_name = 'visual_tempo';
if ((not self.hasFeature(feature_name)) or force_recompute):
vbe = self.getFeature('local_rhythmic_saliency');
# assert librosa.__version__ == '0.7.1'
# result = librosa.beat.beat_track(onset_envelope=vbe, sr=self.sampling_rate, hop_length=1, **kwargs);
result = librosa.beat.beat_track(onset_envelope=vbe, sr=self.sampling_rate, hop_length=1, **kwargs);
self.setFeature(name=feature_name, value=result, params=kwargs);
return self.getFeature(feature_name);
def getVisualTempogram(self, window_length=None, force_recompute=None, norm_columns = None, **kwargs):
"""
:param self:
:param window_length: in seconds
:param force_recompute:
:param kwargs:
:return:
"""
feature_name = 'visual_tempogram';
if ((not self.hasFeature(feature_name)) or force_recompute):
if(window_length is None):
window_length = DEFAULT_TEMPOGRAM_WINDOW_SECONDS;
params = kwargs;
params.update({'force_recompute': force_recompute});
vbe = self.computeImpactEnvelope(cut_suppression_seconds = None);
onset_envelope = vbe;
win_length = int(round(window_length * self.sampling_rate));
sr = self.sampling_rate;
hop_length = 1;
center=kwargs.get('center');
if(center is None):
center=True;
window='hann'
norm=np.inf;
ac_window = librosa.filters.get_window(window, win_length, fftbins=True)
# Center the autocorrelation windows
n = len(onset_envelope)
if center:
onset_envelope = np.pad(onset_envelope, int(win_length // 2),
mode='linear_ramp', end_values=[0, 0])
# Carve onset envelope into frames
odf_frame = librosa.util.frame(onset_envelope,
frame_length=win_length,
hop_length=hop_length)
# Truncate to the length of the original signal
if center:
odf_frame = odf_frame[:, :n]
odf_frame = librosa.util.normalize(odf_frame,axis=0,norm=norm)
if(norm_columns is None):
norm_columns = True;
if(norm_columns):
# Window, autocorrelate, and normalize
result = librosa.util.normalize(
librosa.core.audio.autocorrelate(odf_frame * ac_window[:, np.newaxis], axis=0), norm=norm, axis=0);
else:
result = librosa.core.audio.autocorrelate(odf_frame * ac_window[:, np.newaxis], axis=0);
result = np.true_divide(result, np.max(result.ravel()));
tempo_bpms = librosa.tempo_frequencies(result.shape[0], hop_length=hop_length, sr=sr)
self.setFeature(name='tempogram_bpms', value=tempo_bpms);
###########
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getVisibleImpactEnvelope(self, force_recompute=False, **kwargs):
feature_name = 'impact_envelope';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
result = self.computeImpactEnvelope(forward=True, backward = False, **kwargs);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getForwardVisibleImpactEnvelope(self, force_recompute=False, **kwargs):
"""
Same as getVisibleImpactEnvelope by default.
:param self:
:param force_recompute:
:param kwargs:
:return:
"""
feature_name = 'forward_visual_impact_envelope';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
result = self.computeImpactEnvelope(forward=True, backward = False, **kwargs);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getBackwardVisibleImpactEnvelope(self, force_recompute=False, **kwargs):
feature_name = 'backward_visual_impact_envelope';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
result = self.computeImpactEnvelope(forward=False, backward = True, **kwargs);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getBothWayVisibleImpactEnvelope(self, force_recompute=False, **kwargs):
feature_name = 'both_way_visual_impact_envelope';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
result = self.computeImpactEnvelope(forward=True, backward = True, **kwargs);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getVisibleImpactEnvelopePowers(self, force_recompute=False, **kwargs):
feature_name = 'impact_envelope_powers';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
result0 = self.computeImpactEnvelope(power= 0, **params);
result = np.zeros([len(result0), 3]);
result[:, 0] = result0;
result[:, 1] = self.computeImpactEnvelope(power= 1, **params);
result[:, 2] = self.computeImpactEnvelope(power=2, **params);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getCutTimes(self):
return Event.ToStartTimes(self.getCutEvents());
def getCutEvents(self, force_recompute=False, **kwargs):
"""
Hacky estimate of cuts in a video
:param self:
:param force_recompute:
:param kwargs:
:return:
"""
feature_name = 'cut_events';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
powers = self.getFeature('directogram_powers');
im0 = powers[:, :, 0];
im2 = powers[:, :, 2];
imc = np.true_divide(im2, 0.05 + im0);
medsig = np.median(imc, 0);
medall = np.median(imc);
cutsig = np.true_divide(medsig, medall)
cut_detection_ratio = kwargs.get('cut_detection_ratio');
if(cut_detection_ratio is None):
cut_detection_ratio=CUT_DETECTION_RATIO;
clipsig = (cutsig > cut_detection_ratio).astype(float);
clip_floorsig = (cutsig > CUT_DETECTION_FLOOR).astype(float);
clipsig = np.multiply(clipsig, clip_floorsig);
einds = np.flatnonzero(clipsig);
etimes = einds * truediv(1.0, self.sampling_rate);
ev = Event.FromStartTimes(etimes, type='cut');
ev = self.visualBeatsFromEvents(ev);
evout = []
for e in range(len(ev)):
ev[e].setInfo('frame', einds[e]);
result = ev;
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def visualBeatsFromEvents(self, events):
def downsample_hist(sig, levels):
nperbin = np.power(2, levels);
rshp = sig.reshape(-1, nperbin);
return rshp.sum(axis=1);
if(self.hasFeature('impact_envelope')):
svbe=self.getFeature('impact_envelope');
else:
svbe=self.computeImpactEnvelope(cut_suppression_seconds = None);
flow_powers = self.getFeature('directogram_powers');
vis_tempogram = self.getFeature('visual_tempogram');
vbeats = [];
for e in events:
b = VisualBeat.FromEvent(e);
ei = int(round(b.start*self.sampling_rate*VB_UPSAMPLE_FACTOR));
b.weight = svbe[ei];
histsize = int(128/int(np.power(2,HISTOGRAM_DOWNSAMPLE_LEVELS)))
histslice = np.zeros([histsize, HISTOGRAM_FRAMES_PER_BEAT]);
histslice = np.squeeze(np.mean(histslice, 1));
b.flow_histogram = downsample_hist(sig=histslice, levels=HISTOGRAM_DOWNSAMPLE_LEVELS);
b.flow_histogram = np.divide(b.flow_histogram, np.sum(b.flow_histogram));
b.local_autocor = vis_tempogram[:,ei];
b.local_autocor = b.local_autocor/np.max(b.local_autocor);
b.sampling_rate=self.sampling_rate;
vbeats.append(b);
return vbeats;
def getVisualBeatTimes(self, **kwargs):
return Event.ToStartTimes(self.getVisualBeats(**kwargs));
def getDirectionalFlux(self,
f_sigma=None,
median_kernel=None,
power=None,
**kwargs):
"""
The visual impact complement of a spectral flux matrix
:param self:
:param f_sigma: sigma for the gaussian used to filter, using sp.ndimage.filters.gaussian_filter
:param median_kernel: used in sp.signal.medfilt
:param power: Which power of the flow to use. Usually use 1.
:param kwargs:
:return:
"""
def d_x(im):
d_im = np.zeros(im.shape);
d_im[:, 0] = im[:, 0];
for c in range(1, im.shape[1]):
d_im[:, c] = im[:, c] - im[:, c - 1];
return d_im;
if (f_sigma is None):
f_sigma = [5, 3];
if (median_kernel is None):
median_kernel = [3, 3];
if(power is None):
power = 1;
powers = self.getFeature('directogram_powers');
im = powers[:,:,power].copy();
if (f_sigma is not None):
im = sp.ndimage.filters.gaussian_filter(input=im, sigma=f_sigma, order=0);
im = sp.signal.medfilt(im, median_kernel);
return d_x(im);
def computeImpactEnvelope(self,
forward=True,
backward = False,
f_sigma=None,
median_kernel=None,
highpass_window_seconds= 0.8,
cut_percentile=99,
power=None,
crop = None,
normalize = True,
**kwargs):
"""
:param self:
:param forward: include impact going forward in time
:param backward: include impact going backward in time
:param f_sigma: sigma for the gaussian used to filter, using sp.ndimage.filters.gaussian_filter
:param median_kernel: used in sp.signal.medfilt
:param highpass_window_seconds: highpass window size in seconds
:param cut_percentile: percentile above which to consider cuts and to clip
:param power: Which power of the flow to use. Usually use 1.
:param crop:
:param kwargs:
:return:
"""
upsample_factor = kwargs.get('upsample_factor');
inputargs = dict(f_sigma=f_sigma,
median_kernel=median_kernel,
power=power,
crop = crop);
inputargs.update(kwargs);
im_d = self.getDirectionalFlux(**inputargs);
if(forward and backward):
im = np.fabs(im_d);
elif(forward):
im = -im_d;
im = np.clip(im, 0, None)
elif(backward):
im = im_d;
im = np.clip(im, 0, None)
else:
assert(False), "Must be at least one of either forward or backward."
vimpact = np.squeeze(np.mean(im, 0));
sampling_rate = self.sampling_rate;
if(upsample_factor is not None and (upsample_factor>1)):
newlen = upsample_factor * len(vimpact);
sampling_rate = upsample_factor*sampling_rate;
vimpact = sp.signal.resample(vimpact, newlen);
if(highpass_window_seconds):
order = kwargs.get('highpass_order');
if(order is None):
order = 5;
cutoff = truediv(1.0, highpass_window_seconds);
normal_cutoff = cutoff / (sampling_rate*0.5);
b, a = sp.signal.butter(order, normal_cutoff, btype='high', analog=False)
vimpact = sp.signal.filtfilt(b, a, vimpact);
normfactor = np.max(np.fabs(vimpact[:]));
if (cut_percentile is not None):
fx = np.fabs(vimpact);
pv = np.percentile(fx, cut_percentile);
pvlow = np.percentile(fx, cut_percentile-1);
normfactor = pv;
ptile = (vimpact > pv).astype(float);
pntile = (vimpact < -pv).astype(float);
pboth = ptile+pntile;
einds = np.flatnonzero(pboth);
lastind = -2;
for j in range(len(einds)):
if(einds[j]==(lastind+1)):
vimpact[einds[j]]=0;
else:
vimpact[einds[j]]=pvlow;
if(normalize):
vimpact = np.true_divide(vimpact, normfactor);
return vimpact;
# 0.8, cut_suppression_seconds = 0.4,
def computeImpactEnvelopeOld(self, f_sigma=None, median_kernel=None, highpass_window_seconds= 0.8, cut_percentile=99, power=None, crop = None, normalize=None, **kwargs):
"""
I believe this is the version of the function that I used in the original paper. Keeping it around for record.
:param self:
:param f_sigma:
:param median_kernel:
:param highpass_window_seconds:
:param cut_percentile:
:param power:
:param crop:
:param normalize:
:param kwargs:
:return:
"""
def d_x(im):
# d_im = np.zeros(im.shape, dtype=np.float128);
d_im = np.zeros(im.shape);
d_im[:, 0] = im[:, 0];
for c in range(1, im.shape[1]):
d_im[:, c] = im[:, c] - im[:, c - 1];
return d_im;
if (f_sigma is None):
f_sigma = [5, 3];
if (median_kernel is None):
median_kernel = [3, 3];
if(power is None):
power = 1;
upsample_factor = kwargs.get('upsample_factor');
powers = self.getFeature('directogram_powers');
print(("computing impact env with power {}".format(power)));
im = powers[:,:,power].copy();
if (f_sigma is not None):
im = sp.ndimage.filters.gaussian_filter(input=im, sigma=f_sigma, order=0);
im = sp.signal.medfilt(im, median_kernel);
im = -d_x(im);
im = np.clip(im, 0, None)
svbe = np.squeeze(np.mean(im, 0));
sampling_rate = self.sampling_rate;
if(upsample_factor is not None and (upsample_factor>1)):
newlen = upsample_factor * len(svbe);
sampling_rate = upsample_factor*sampling_rate;
svbe = sp.signal.resample(svbe, newlen);
if(highpass_window_seconds):
order = kwargs.get('highpass_order');
if(order is None):
order = 5;
cutoff = truediv(1.0, highpass_window_seconds);
normal_cutoff = cutoff / (sampling_rate*0.5);
b, a = sp.signal.butter(order, normal_cutoff, btype='high', analog=False)
svbe = sp.signal.filtfilt(b, a, svbe);
normfactor = np.max(np.fabs(svbe[:]));
if (cut_percentile is not None):
fx = np.fabs(svbe);
pv = np.percentile(fx, cut_percentile);
pvlow = np.percentile(fx, cut_percentile-1);
normfactor = pv;
ptile = (svbe > pv).astype(float);
pntile = (svbe < -pv).astype(float);
pboth = ptile+pntile;
einds = np.flatnonzero(pboth);
lastind = -2;
for j in range(len(einds)):
if(einds[j]==(lastind+1)):
svbe[einds[j]]=0;
else:
svbe[einds[j]]=pvlow;
if(normalize is not False):
svbe = np.true_divide(svbe, normfactor);
return svbe;
def getVisibleImpacts(self, force_recompute=False, include_cut_events = None, **kwargs):
feature_name = 'visible_impacts';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
svbe = self.getFeature('impact_envelope', **kwargs);
upsample_factor = kwargs.get('upsample_factor');
if(upsample_factor is None):
upsample_factor = 1;
u_sampling_rate = self.sampling_rate*upsample_factor;
peak_params = self._getDefaultPeakPickingTimeParams();
peak_params.update(kwargs); # if params given in arguments, those will override the defaults here.
v_events = Event.FromSignalPeaks(signal=svbe, sampling_rate=u_sampling_rate, **peak_params);
if(include_cut_events):
cut_events = self.getFeature('cut_events');
v_events = v_events + cut_events;
Event.Sort(v_events);
result = self.visualBeatsFromEvents(v_events);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getForwardVisibleImpacts(self, force_recompute=False, include_cut_events = None, **kwargs):
feature_name = 'forward_visual_beats';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
local_saliency = self.getFeature('forward_visual_impact_envelope', **kwargs);
upsample_factor = kwargs.get('upsample_factor');
if(upsample_factor is None):
upsample_factor = 1;
u_sampling_rate = self.sampling_rate*upsample_factor;
v_events = Event.FromSignalPeaks(signal=local_saliency, sampling_rate=u_sampling_rate, event_type= 'forward', **kwargs);
if(include_cut_events):
cut_events = self.getFeature('cut_events');
v_events = v_events + cut_events;
Event.Sort(v_events);
result = Event.SetDirections(v_events, direction=Event.DIRECTION_FORWARD);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def getBackwardVisibleImpacts(self, force_recompute=False, include_cut_events = None, **kwargs):
feature_name = 'backward_visual_beats';
if ((not self.hasFeature(feature_name)) or force_recompute):
params = kwargs;
local_saliency = self.getFeature('backward_visual_impact_envelope', **kwargs);
upsample_factor = kwargs.get('upsample_factor');
if(upsample_factor is None):
upsample_factor = 1;
u_sampling_rate = self.sampling_rate*upsample_factor;
v_events = Event.FromSignalPeaks(signal=local_saliency, sampling_rate=u_sampling_rate, **kwargs);
if(include_cut_events):
cut_events = self.getFeature('cut_events');
v_events = v_events + cut_events;
Event.Sort(v_events);
# result = self.visualBeatsFromEvents(v_events);
result = Event.SetDirections(v_events, direction=Event.DIRECTION_BACKWARD);
self.setFeature(name=feature_name, value=result, params=params);
return self.getFeature(feature_name);
def findAccidentalDanceSequences(self, target_n_beats = 7, n_samples=25, delta_range = None):
if(delta_range is None):
delta_range = [0.02, 0.5];
deltas = np.linspace(delta_range[0], delta_range[1], num=n_samples, endpoint=True);
deltapick = delta_range[0];
sequences = [];
for i, delta in enumerate(deltas):
peak_vars = self._getDefaultPeakPickingTimeParams(delta=delta);
sequences = self.getVisualBeatSequences(peak_vars=peak_vars, print_summary=False);
# print("Delta {} has top sequence with {} beats".format(delta, len(sequences[0])));
if(len(sequences[0])<=target_n_beats):
deltapick = delta;
break;
print(("Selected delta value {}".format(deltapick)));
return sequences;
def getVisualBeatSequences(self,
search_tempo=None,
target_period=None,
search_window=0.75,
min_beat_limit=None,
max_beat_limit=None,
unary_weight=None,
binary_weight=None,
break_on_cuts=None,
peak_vars=None,
time_range=None,
n_return = None,
unsorted = False,
print_summary = True,
**kwargs):
"""
:param self:
:param search_tempo: optional tempo you would like to find visual beats at
:param target_period: optional target period you would like to use for finding beats. Ignored if search tempo is provided.
:param search_window: longest amount of time (seconds) allowed between beats before a segment is broken into multiple segments
:param min_beat_limit: only consider sequences with
:param unary_weight:
:param binary_weight:
:param break_on_cuts:
:param peak_vars:
:param kwargs:
:return:
"""
if (peak_vars is not None):
# impacts = self.getFeature('visible_impacts', force_recompute=True, **peak_vars);
impacts = self.getVisualBeats(force_recompute = True, **peak_vars);
else:
# impacts = self.getFeature('visible_impacts');
impacts = self.getVisualBeats();
if(time_range is not None):
impactseg = [];
for i in impacts:
if(i.start>time_range[0] and i.start < time_range[1]):
impactseg.append(i);
impacts = impactseg;
if (search_tempo is not None):
tempo = search_tempo;
beat_time = np.true_divide(60.0, tempo);
sequences = VisualBeat.PullOptimalPaths_Basic(impacts, target_period=beat_time, unary_weight=unary_weight,
binary_weight=binary_weight, break_on_cuts=break_on_cuts,
window_size=search_window);
elif(target_period is not None):
sequences = VisualBeat.PullOptimalPaths_Basic(impacts, target_period=target_period, unary_weight=unary_weight,
binary_weight=binary_weight, break_on_cuts=break_on_cuts,
window_size=search_window);
else:
sequences = VisualBeat.PullOptimalPaths_Autocor(impacts, unary_weight=unary_weight, binary_weight=binary_weight,
break_on_cuts=break_on_cuts, window_size=search_window);
r_sequences = [];
if(min_beat_limit is None):
min_beat_limit = 2;
if(max_beat_limit is None):
max_beat_limit = len(impacts)+1;
for S in sequences:
if ((len(S) > min_beat_limit) and (len(S) < max_beat_limit)):
r_sequences.append(S);
if(not unsorted):
r_sequences.sort(key=len, reverse=True);
if(n_return is not None):
r_sequences = r_sequences[:n_return];
if(print_summary):
print(("{} segments".format(len(r_sequences))));
for s in range(len(r_sequences)):
print(("Segment {} has {} beats".format(s, len(r_sequences[s]))));
return r_sequences;
def printVisualBeatSequences(self,
search_tempo=None,
target_period=None,
search_window=None,
min_beat_limit=None,
max_beat_limit=None,
unary_weight=None,
binary_weight=None,
break_on_cuts=None,
peak_vars=None,
n_return = None,
time_range=None, **kwargs):
"""
:param self:
:param target_period:
:param search_tempo:
:param search_window:
:param beat_limit:
:param unary_weight:
:param binary_weight:
:param break_on_cuts:
:param n_return:
:param peak_vars:
:param time_range:
:param kwargs:
:return:
"""
sorted = True;
sequence_args = dict(
search_tempo=search_tempo,
target_period=target_period,
search_window=search_window,
min_beat_limit=min_beat_limit,
max_beat_limit=max_beat_limit,
unary_weight=unary_weight,
binary_weight=binary_weight,
break_on_cuts=break_on_cuts,
peak_vars=peak_vars,
n_return=n_return,
time_range=time_range);
seqs = self.getVisualBeatSequences(**sequence_args)
print(("sequence arguments were:\n{}".format(sequence_args)));
print(("There were {} sequences total".format(len(seqs))));
nclips = 0;
rsegments = [];
for S in seqs:
if (len(S) > 1):
nclips = nclips + 1;
rsegments.append(S);
# rsegments.sort(key=len, reverse=True);
# if (n_return is not None):
# rsegments = rsegments[:n_return];
Event.PlotSignalAndEvents(self.getFeature('impact_envelope'), sampling_rate=self.sampling_rate*VB_UPSAMPLE_FACTOR, events=rsegments[0], time_range=time_range);
return rsegments;
def plotEvents(self, events, time_range = 'default', **kwargs):
time_range_use = time_range;
if(time_range.lower() == 'default'):
time_range_use = [0,0];
time_range_use[0] = events[0].start-1;
time_range_use[1] = events[-1].start+1;
signal = self.getFeature('local_rhythmic_saliency');
mplt = Event.PlotSignalAndEvents(signal, sampling_rate=self.sampling_rate * VB_UPSAMPLE_FACTOR, events=events, time_range=time_range_use, **kwargs);
plt.xlabel('Time (s)')
return mplt;
def plotCutEvents(self, **kwargs):
signal = self.getFeature('impact_envelope');
events = self.getFeature('cut_events', **kwargs);
Event.PlotSignalAndEvents(signal, sampling_rate=self.sampling_rate*VB_UPSAMPLE_FACTOR, events=events, **kwargs);
def plotVisibleImpacts(self, **kwargs):
signal = self.getFeature('impact_envelope');
events = self.getFeature('visible_impacts', **kwargs);
Event.PlotSignalAndEvents(signal, sampling_rate=self.sampling_rate*VB_UPSAMPLE_FACTOR, events=events, **kwargs);
plt.title('Impact Envelope & Visual Beats')
plt.xlabel('Time (s)')
plt.ylabel('Impact Strength')
def plotImpactEnvelope(self, **kwargs):
signal = self.getFeature('local_rhythmic_saliency');
# events = self.getFeature('visual_beats', **kwargs);
events = None;
Event.PlotSignalAndEvents(signal, sampling_rate=self.sampling_rate*VB_UPSAMPLE_FACTOR, events=events, **kwargs);
plt.title('Impact Envelope & Visual Beats')
plt.xlabel('Time (s)')
plt.ylabel('Impact Strength')
def plotVisualBeats(self, **kwargs):
signal = self.getFeature('local_rhythmic_saliency');
events = self.getFeature('visual_beats', **kwargs);
Event.PlotSignalAndEvents(signal, sampling_rate=self.sampling_rate*VB_UPSAMPLE_FACTOR, events=events, **kwargs);
plt.title('Impact Envelope & Visual Beats')
plt.xlabel('Time (s)')
plt.ylabel('Impact Strength')
def loadFlowFeatures(self):
self.load(features_to_load=['directogram_powers', 'directogram']);
FEATURE_FUNCS['local_rhythmic_saliency'] = getLocalRhythmicSaliency;
FEATURE_FUNCS['directogram_powers'] = getDirectogramPowers;
FEATURE_FUNCS['directogram'] = getDirectogram;
FEATURE_FUNCS['impact_envelope'] = getVisibleImpactEnvelope;
FEATURE_FUNCS['impact_envelope_powers'] = getVisibleImpactEnvelopePowers;
FEATURE_FUNCS['visible_impacts'] = getVisibleImpacts;
FEATURE_FUNCS['visual_beats'] = getVisualBeats;
# FEATURE_FUNCS['backward_visual_beats'] = getBackwardVisualBeats;
# FEATURE_FUNCS['forward_visual_beats'] = getForwardVisualBeats;
FEATURE_FUNCS['backward_visual_impact_envelope'] = getBackwardVisibleImpactEnvelope;
FEATURE_FUNCS['both_way_visual_impact_envelope'] = getBothWayVisibleImpactEnvelope;
FEATURE_FUNCS['forward_visual_impact_envelope'] = getForwardVisibleImpactEnvelope;
FEATURE_FUNCS['directional_flux'] = getDirectionalFlux;
FEATURE_FUNCS['visual_tempogram'] = getVisualTempogram;
FEATURE_FUNCS['cut_events']=getCutEvents;
================================================
FILE: src/video2npz/visbeat3/visbeat3/VisBeatDefines.py
================================================
VB_DEBUG = True;
DEFAULT_TEMPOGRAM_WINDOW_SECONDS = 5;
AUDIO_DEFAULT_HOP_LENGTH = 512;
================================================
FILE: src/video2npz/visbeat3/visbeat3/VisBeatExampleVideo.py
================================================
import os
from visbeat3.SourceLocationParser import ParseSourseLocation
class VisBeatExampleVideo(object):
def __init__(self, name, url, start_beat=None, end_beat = None, display_name = None, code=None, leadin=None, **kwargs):
self.name = name;
self.url = url;
self.start_beat = start_beat;
self.end_beat = end_beat;
self._display_name = display_name;
self._code = code;
self.leadin = leadin;
for k in kwargs:
setattr(self, k, kwargs[k]);
if(code is None):
sloc = ParseSourseLocation(self.url);
self._code = sloc.code;
#
@property
def code(self):
return self._getCode();
def _getCode(self):
return self._code;
@code.setter
def code(self, value):
self._setCode(value);
def _setCode(self, value):
self._code = value;
#
# def _ytEmbedCode(self):
#
@property
def display_name(self):
return self._getDisplayName();
def _getDisplayName(self):
if(self._display_name is None):
return self.name;
else:
return self._display_name;
#
def _ytWatchURL(self):
return 'https://www.youtube.com/watch?v={}'.format(self.code);
def _ytEmbedURL(self, autoplay=None):
s = 'https://www.youtube.com/embed/{}'.format(self.code);
if(autoplay):
s = s+'?autoplay=1';
return s;
def _ytThumbURL(self):
return 'https://ytimg.googleusercontent.com/vi/{}/default.jpg'.format(self.code);
def _fancyBoxCode(self, with_label = None):
""""""
s = HTMLCode();
if(with_label):
s.addLine("")
s.addLine(''.format(watchurl=self._ytWatchURL(),
alt_name = self.display_name,
thumburl = self._ytThumbURL()))
if(with_label):
s.addLine('');
# s.addLine('{displayname}'.format(self.url, self.display_name));
s.addLine('{displayname}'.format(displayname=self.display_name));
s.addLine(' ')
s.addLine(' ');
return s.string;
from bs4 import BeautifulSoup
class HTMLCode(object):
def __init__(self, start_string=None):
self._code = "";
self._lines = [];
if(start_string is not None):
self.add(start_string)
self._soup = None;
#
@property
def string(self):
return self._getString();
def _getString(self):
return BeautifulSoup(self.code).prettify();
#
@property
def code(self):
return self._getCode();
def _getCode(self):
return self._code;
@code.setter
def code(self, value):
self._setCode(value);
def _setCode(self, value):
self._code = value;
#
#
@property
def soup(self):
return self._getSoup();
def _getSoup(self):
if(self._soup is None):
self._makeSoup();
return self._soup;
def _makeSoup(self):
self._soup = BeautifulSoup(self.code, 'html.parser');
# @soup.setter
# def soup(self, value):
# self._setSoup(value);
# def _setSoup(self, value):
# self._soup = value;
#
def add(self, s):
self.code = self.code+s;
def addLine(self, s):
self.code = self.code+'\n'+s;
def startTable(self, id=None, class_ = None, **kwargs):
targs = dict(table_width="80%", border=1, cellspacing=1, cellpadding=1)
targs.update(kwargs);
s = ''.format(**targs);
self.addLine(s);
self.addLine('');
def endTable(self):
self.addLine("");
self.addLine("
");
def startRow(self):
self.addLine('');
def endRow(self):
self.addLine(' ');
def addColumnLabel(self, label):
self.addLine('')
self.addLine(label);
self.addLine(' ');
def addRowLabel(self, label):
self.addLine('')
self.addLine(label);
self.addLine(' ');
def addRowCell(self, content):
self.addLine("");
self.addLine("")
self.addLine(content);
self.addLine("")
self.addLine(" ");
================================================
FILE: src/video2npz/visbeat3/visbeat3/VisBeatImports.py
================================================
from .VisBeatDefines import *
from .AImports import *
from .AObject import AObject
import numpy as np
import scipy as sp
import os
import imageio
import matplotlib
try:
import matplotlib.pyplot as plt
except ImportError as e:
AWARN("matplotlib problem... if you are using conda try installing with 'conda install matplotlib'")
matplotlib.use('agg');
import matplotlib.pyplot as plt
import matplotlib.style as ms
import io
import base64
import math
from operator import truediv
import time
import shutil
from time import gmtime, strftime, localtime
import librosa;
from ._mediafiles import GetVBMarkPath
def local_time_string():
return strftime("%Y-%m-%d_%H:%M:%S", localtime());
def VBWARN(message):
print(message)
#
# def send_warnings_to_print_red(message, category, filename, lineno):
# print(colored('{} WARNING! file: {} Line:{}\n{}'.format(category, filename, lineno, message), 'red'))
# old_showwarning = warnings.showwarning
# warnings.showwarning = send_warnings_to_print_red;
VB_MACHINE_ID = None;
# if(VB_MACHINE_ID):
# matplotlib.use('PS');
ISNOTEBOOK = False;
if(runningInNotebook()):
ISNOTEBOOK = True;
import IPython;
import IPython.display
ms.use('seaborn-muted')
if(not runningInSpyder()):
get_ipython().magic('matplotlib inline')
from IPython.lib import kernel
connection_file_path = kernel.get_connection_file()
connection_file = os.path.basename(connection_file_path)
kernel_id = connection_file.split('-', 1)[1].split('.')[0]
# print("Kernel ID:\n{}".format(kernel_id));
from IPython.display import HTML
VBIPY = IPython;
#%matplotlib inline
# else:
# matplotlib.use('PS');
def vb_get_ipython():
return VBIPY;
def clipping_params(clip_bins=30, clip_fraction=0.95):
return dict(clip_bins=clip_bins, clip_fraction=clip_fraction);
def get_hist_clipped(signal, clip_bins=30, clip_fraction=0.95):
holdshape = signal.shape[:];
sigrav = signal.copy().ravel();
sigh, sigb = np.histogram(sigrav, bins=clip_bins);
maxbini=np.argmax(sigh);
totalmass = np.sum(sigh);
total_included = truediv(sigh[maxbini],totalmass);
prevbini = maxbini;
nextbini=maxbini;
bins_included = 1;
icounter=0;
while(total_included=0 and sigh[prevbini]==0) and (nextbini<(clip_bins) and sigh[nextbini]==0)):
prevbini=prevbini-1;
nextbini=nextbini+1;
else:
if((prevbini>=0 and sigh[prevbini]>0) or nextbini>=clip_bins):
prevbini=prevbini-1;
if((nextbini<(clip_bins) and sigh[nextbini]>0) or prevbini<0):
nextbini=nextbini+1;
included_segment=sigh[max(prevbini,0):min(nextbini+1, clip_bins)];
total_included = truediv(np.sum(included_segment), totalmass);
bins_included=len(included_segment);
icounter+=1;
clipsig = np.clip(a=sigrav, a_min=sigb[max(prevbini,0)], a_max=sigb[min(nextbini,clip_bins)]);
clipsig.shape=signal.shape
return clipsig;
def np_scale_to_range(data, value_range=None):
if(value_range is None):
value_range = [0.0,255.0];
d = data.copy().ravel();
currentmin=np.min(d);
currentmax=np.max(d);
currentscale = currentmax-currentmin;
if(currentscale==0):
VBWARN("CANNOT SCALE CONSTANT ARRAY TO RANGE")
return;
divscale = truediv(1.0,currentscale);
newrange=(value_range[1]-value_range[0]);
d = (d*divscale)*newrange;
newmin = (currentmin*divscale)*newrange;
d = d-newmin+value_range[0]
d.shape=data.shape;
return d
================================================
FILE: src/video2npz/visbeat3/visbeat3/VisualBeat.py
================================================
from .Event import *
class VisualBeat(Event):
def VBOBJECT_TYPE(self):
return 'VisualBeat';
def __init__(self, start=None, type=None, weight=None, index=None, unrolled_start = None, direction=0):
Event.__init__(self, start=start, type=type, weight=weight, index=index, unrolled_start=unrolled_start, direction=direction);
def initializeBlank(self):
Event.initializeBlank(self);
self.flow_histogram = None;
self.local_autocor = None;
self.sampling_rate = None;
def __str__(self):
return "start:{}\ntype:{}\nweight:{}\nindex:{}\nunrolled_start:{}\nis_active:{}\n".format(self.start, self.type, self.weight, self.index, self.unrolled_start, self.is_active);
# def toGUIDict(self, is_active=1):
# return dict(start=self.start, index=self.index, is_active=None)
def toDictionary(self):
d=Event.toDictionary(self);
# d['start']=self.start;
# d['type']=self.type;
# d['weight']=self.weight;
# d['index']=self.index;
# d['unrolled_start']=self.unrolled_start;
d['flow_histogram']=self.flow_histogram;
d['local_autocor']=self.local_autocor;
d['sampling_rate']=self.sampling_rate;
return d;
def initFromDictionary(self, d):
Event.initFromDictionary(self, d);
self.start = d['start'];
self.type = d['type'];
self.weight = d['weight'];
self.index = d['index'];
self.unrolled_start = d['unrolled_start'];
self.flow_histogram = d.get('flow_histogram');
self.local_autocor = d.get('local_autocor');
self.sampling_rate = d.get('sampling_rate');
def clone(self, start=None):
if(start):
newv = VisualBeat(start = start, type=self.type, weight=self.weight, index=self.index);
else:
newv = VisualBeat(start = self.start, type=self.type, weight=self.weight, index=self.index);
newv.flow_histogram = self.flow_histogram.copy();
newv.local_autocor = self.local_autocor.copy();
newv.sampling_rate = self.sampling_rate;
return newv;
@staticmethod
def FromEvent(e):
if(isinstance(e,VisualBeat)):
return e.clone();
else:
return VisualBeat(start=e.start, type=e.type, weight=e.weight, index=e.index);
@staticmethod
def time_window_func(max_separation, break_on_cuts=None):
def window_func(a, b):
if(break_on_cuts and (a.type=='cut' or b.type=='cut')):
return False;
if(np.fabs(a.start-b.start)3.75):
score=-1;
return binary_weight*score;
return objective_func;
@staticmethod
def angle_binary_objective(binary_weight=None, absolute=None):
if (binary_weight is None):
binary_weight = 1.0;
# if (angle_weight is None):
# angle_weight = 0.0;
def objective_func(a, b):
# T = np.fabs(a.start - b.start);
# tempo_score = -np.power(np.log(truediv(T, target_period)), 2.0) * binary_weight;
# should mask out unary contribution from orthogonal angles
if(absolute):
return binary_weight * (np.fabs(np.dot(a.flow_histogram, b.flow_histogram)) - 0.70710678118); # cos 45 degrees
else:
return binary_weight * (np.dot(a.flow_histogram, b.flow_histogram)); # cos 45 degrees
return objective_func;
@staticmethod
def Double(events, type=None):
doubled = [];
for e in range(1, len(events)):
halfstart = 0.5 * (events[e].start + events[e - 1].start);
newhevent = events[e].clone(start=halfstart);
if(type is not None):
newhevent.type = type;
doubled.append(newhevent);
doubled.append(events[e]);
return doubled;
@staticmethod
def weight_unary_objective(unary_weight=None):
if(unary_weight is None):
unary_weight = 1.0;
def getweight_func(b):
return unary_weight*b.weight;
return getweight_func;
@staticmethod
def PullOptimalPaths_Basic(vis_beats, target_period, unary_weight=None, binary_weight=None, window_size=None, break_on_cuts = None):
if(window_size is None):
window_size = DEFAULT_WINDOW_FACTOR*target_period;
binary_objective = VisualBeat.tempo_binary_objective(target_period=target_period, binary_weight = binary_weight);
unary_objective = VisualBeat.weight_unary_objective(unary_weight=unary_weight);
window_function = VisualBeat.time_window_func(max_separation = window_size, break_on_cuts=break_on_cuts);
return VisualBeat.DynamicProgramOptimalPaths(vis_beats=vis_beats,
unary_objective_func=unary_objective,
binary_objective_func=binary_objective,
window_func=window_function);
@staticmethod
def PullOptimalPaths(vis_beats, unary_fn=None, binary_fn=None, window_fn=None, target_period=None, unary_weight=None, binary_weight=None, window_size=None,
break_on_cuts=None):
if (window_size is None):
window_size = DEFAULT_WINDOW_FACTOR * target_period;
if(binary_fn == 'autocor'):
binary_objective = VisualBeat.autocor_binary_objective(binary_weight=binary_weight);
elif(binary_fn == 'angle'):
binary_objective = VisualBeat.angle_binary_objective(binary_weight=binary_weight);
else:
binary_objective = VisualBeat.tempo_binary_objective(target_period=target_period, binary_weight=binary_weight);
unary_objective = VisualBeat.weight_unary_objective(unary_weight=unary_weight);
window_function = VisualBeat.time_window_func(max_separation=window_size, break_on_cuts=break_on_cuts);
return VisualBeat.DynamicProgramOptimalPaths(vis_beats=vis_beats,
unary_objective_func=unary_objective,
binary_objective_func=binary_objective,
window_func=window_function);
@staticmethod
def PullOptimalPaths_Autocor(vis_beats, unary_weight=None, binary_weight=None, window_size=None,
break_on_cuts=None, **kwargs):
if (window_size is None):
# assert(False), 'no window size provided'
# window_size = DEFAULT_WINDOW_FACTOR * target_period;
window_size = 200;
AWARN('NO WINDOWSIZE PROVIDED! PullOptimalPaths_Autocor');
binary_objective = VisualBeat.autocor_binary_objective(binary_weight=binary_weight);
unary_objective = VisualBeat.weight_unary_objective(unary_weight=unary_weight);
window_function = VisualBeat.time_window_func(max_separation=window_size, break_on_cuts=break_on_cuts);
return VisualBeat.DynamicProgramOptimalPaths(vis_beats=vis_beats,
unary_objective_func=unary_objective,
binary_objective_func=binary_objective,
window_func=window_function);
@staticmethod
def DynamicProgramOptimalPaths(vis_beats, unary_objective_func, binary_objective_func, window_func):
class Node(object):
def __init__(self, object, prev_node=None):
self.object = object;
self.prev_node = prev_node;
self.cum_score=None;
nodes = [];
beats = Event.GetSorted(vis_beats);
Event.ApplyIndices(beats);
for b in beats:
nodes.append(Node(object=b));
nodes[0].prev_node = None;
nodes[0].cum_score = unary_objective_func(nodes[0].object);
current_segment = [];
segments = [];
for n in range(1,len(nodes)):
current_node = nodes[n];
current_segment.append(current_node);
options = [];
j = n-1;
while(j>=0 and window_func(current_node.object,nodes[j].object)):
options.append(nodes[j]);
j = j-1;
if(len(options)==0):
current_node.prev_node=None;
current_node.cum_score=unary_objective_func(current_node.object);
segments.append(current_segment);
current_segment = [];
else:
best_choice = options[0];
best_score = options[0].cum_score+binary_objective_func(current_node.object, best_choice.object);
for o in range(1,len(options)):
score = options[o].cum_score+binary_objective_func(current_node.object, options[o].object);
if(score>best_score):
best_choice=options[o];
best_score=score;
current_node.prev_node = best_choice;
current_node.cum_score = best_score+unary_objective_func(current_node.object);
if(len(current_segment)>0):
segments.append(current_segment);
sequences = [];
for S in segments:
seq = [];
max_node = S[0];
max_score = max_node.cum_score;
for n in range(len(S)):
if(S[n].cum_score>max_score):
max_node = S[n];
max_score = max_node.cum_score;
trace_node = max_node;
while(trace_node.prev_node is not None):
seq.append(trace_node.object);
trace_node=trace_node.prev_node;
seq.reverse();
sequences.append(seq);
return sequences;
================================================
FILE: src/video2npz/visbeat3/visbeat3/Warp.py
================================================
from .VisBeatImports import *
from .EventList import *
import math
DEFAULT_LEAD_TIME = 0;
DEFAULT_TAIL_TIME = 0;
class Warp(AObject):
"""Warp (class): defines how one time signal should be warped to another. Given primarily as source/target events to be matched.
Attributes:
source_events: source events
target_events: target events
"""
def VBOBJECT_TYPE(self):
return 'Warp';
def __init__(self, path=None):
AObject.__init__(self, path=path);
if(path):
self.loadFile();
@staticmethod
def FromEvents(source_events, target_events):
w = Warp();
sevents = source_events;
if(isinstance(source_events, EventList)):
sevents = source_events.events;
tevents = target_events;
if(isinstance(target_events, EventList)):
tevents = target_events.events;
w.source_events=sevents;
w.target_events=tevents;
# w.repeatShorterEvents();
return w
@staticmethod
def FromEventLists(source_eventlist, target_eventlist):
w = Warp();
w.source_events = source_eventlist.events;
w.target_events = target_eventlist.events;
# w.repeatShorterEvents();
return w
def initializeBlank(self):
AObject.initializeBlank(self);
self.source_events = [];
self.target_events = [];
# self.a_info['WarpType'] = 'Linear';
self.warp_func = None; # Warp.LinearInterp;
# self.warp_func_st = None;
# self.warp_func_ts = None;
def getTargetStart(self, lead=None):
target_start = self.target_events[0].getStartTime();
if (lead is None):
lead = min(target_start, DEFAULT_LEAD_TIME);
return target_start - lead;
def getTargetEnd(self, lead=None):
lastind = min(len(self.source_events), len(self.target_events)) - 1;
return self.target_events[lastind].getStartTime() + DEFAULT_TAIL_TIME;
def getSourceStart(self):
source_start = self.source_events[0].getUnrolledStartTime();
return source_start;
def getSourceEnd(self):
lastind = min(len(self.source_events), len(self.target_events)) - 1;
return self.source_events[lastind].getUnrolledStartTime();
def getWarpedSourceStart(self, lead=None):
source_start = self.source_events[0].getUnrolledStartTime();
if (lead is None):
lead = min(source_start, DEFAULT_LEAD_TIME);
return self.warpSourceTime(source_start - lead);
def getWarpedSourceEnd(self, tail=None):
last_event = min(len(self.source_events), len(self.target_events)) - 1;
source_end = self.source_events[last_event].getUnrolledStartTime();
if (tail is None):
tail = DEFAULT_TAIL_TIME;
source_end = source_end + tail;
return self.warpSourceTime(source_end);
def setWarpFunc(self, warp_type, **kwargs):
if (warp_type == 'square'):
self.warp_func = [Warp.SquareInterp, Warp.SquareInterp];
elif (warp_type == 'linear'):
self.warp_func = [Warp.LinearInterp, Warp.LinearInterp];
elif (warp_type == 'cubic'):
self.warp_func = [Warp.CubicInterp, Warp.CubicInterp];
elif (warp_type == 'quad'):
self.warp_func = [
Warp.GetEventWarpFunc(self.source_events, self.target_events, Warp.WFunc_Quadratic(), **kwargs),
Warp.GetEventWarpFunc(self.target_events, self.source_events, Warp.WFunc_Quadratic(), **kwargs)];
elif (warp_type == 'mouth'):
self.warp_func = [
Warp.GetEventWarpFunc(self.source_events, self.target_events, Warp.WFunc_Mouth(**kwargs), **kwargs),
Warp.GetEventWarpFunc(self.target_events, self.source_events, Warp.WFunc_Mouth(**kwargs), **kwargs)];
elif (warp_type == 'weight'):
self.warp_func = [
Warp.GetEventWarpFunc(self.source_events, self.target_events,
Warp.WFunc_Weight(use_to_weights=None, **kwargs), **kwargs),
Warp.GetEventWarpFunc(self.target_events, self.source_events,
Warp.WFunc_Weight(use_to_weights=True, **kwargs), **kwargs)];
elif (warp_type == 'half_accel'):
self.warp_func = [
Warp.GetEventWarpFunc(self.source_events, self.target_events, Warp.WFunc_P(p=0.5),
**kwargs),
Warp.GetEventWarpFunc(self.target_events, self.source_events, Warp.WFunc_P(p=0.5),
**kwargs)];
elif (warp_type == 'p'):
p = kwargs.get('p');
self.warp_func = [
Warp.GetEventWarpFunc(self.source_events, self.target_events, Warp.WFunc_P(p=p),
**kwargs),
Warp.GetEventWarpFunc(self.target_events, self.source_events, Warp.WFunc_P(p=p),
**kwargs)];
elif (warp_type == 'target_time_source_fraction'):
self.warp_func = [
Warp.GetEventWarpFunc(self.source_events, self.target_events,
Warp.WFunc_targettime_sourcefraction(**kwargs),
**kwargs),
Warp.GetEventWarpFunc(self.target_events, self.source_events,
Warp.WFunc_targettime_sourcefraction(**kwargs),
**kwargs)];
elif (warp_type == 'target_source_fractions'):
self.warp_func = [
Warp.GetEventWarpFunc(self.source_events, self.target_events,
Warp.WFunc_target_source_fractions(**kwargs),
**kwargs),
Warp.GetEventWarpFunc(self.target_events, self.source_events,
Warp.WFunc_target_source_fractions(**kwargs),
**kwargs)];
elif (warp_type is not None):
self.warp_func = [warp_type, warp_type];
self.setInfo('WarpType', warp_type);
return;
def warpSourceTime(self, t):
return self.warp_func[0](t, a_events=self.source_events, b_events=self.target_events);
def warpSourceTimes(self, t):
tw = t.copy();
for a in range(len(t)):
tw[a] = self.warpSourceTime(t[a]);
return tw;
def warpTargetTime(self, t):
return self.warp_func[1](t, a_events=self.target_events, b_events=self.source_events);
def warpTargetTimes(self, t):
tw = t.copy();
for a in range(len(t)):
tw[a] = self.warpTargetTime(t[a]);
return tw;
def plot(self, xlim=None, sampling_rate=None, new_figure=None, render_control_points=True, render_labels=True,
time_range=None, full_source_range=None, **kwargs):
if (sampling_rate is None):
sampling_rate = 30;
source_duration = self.getSourceEnd() - self.getSourceStart();
old_frame_time = truediv(1.0, sampling_rate);
target_start = self.getTargetStart();
target_end = self.getTargetEnd();
# if(xlim is not None):
# target_start=xlim[0]
target_duration = target_end - target_start;
new_n_samples = target_duration * sampling_rate;
target_start_times = np.linspace(target_start, target_end, num=new_n_samples, endpoint=False);
unwarped_target_times = [];
for st in target_start_times:
unwarped_target_times.append(self.warpTargetTime(st));
if (new_figure):
fig = plt.figure();
unwarped_target_times = np.array(unwarped_target_times);
# unwarped_target_times = unwarped_target_times-unwarped_target_times[0]+target_start;
plt.plot(target_start_times, unwarped_target_times, '-');
if (render_control_points):
lastind = min(len(self.source_events), len(self.target_events)) - 1;
targeteventtimes = Event.ToStartTimes(self.target_events[:lastind]);
sourceeventtimes = Event.ToStartTimes(self.source_events[:lastind]);
plt.plot(targeteventtimes, sourceeventtimes, 'o', label='Control Points');
if (xlim is not None):
xrng = [xlim[0] + target_start, xlim[1] + target_start];
ylim = [self.warpTargetTime(xrng[0]), self.warpTargetTime(xrng[1])];
plt.xlim(xrng);
plt.ylim(ylim);
if (time_range is not None):
plt.xlim(time_range);
if (render_labels):
plt.ylabel('Source Time');
plt.xlabel('Target Time');
plt.title('Warp Curve')
if (new_figure is not None):
return fig;
def plotImage(self, xlim=None, sampling_rate=None):
if (sampling_rate is None):
sampling_rate = 30;
target_start = self.getTargetStart();
target_end = self.getTargetEnd() + 10;
target_duration = target_end - target_start;
new_n_samples = target_duration * sampling_rate;
target_start_times = np.linspace(target_start, target_end, num=new_n_samples, endpoint=True);
unwarped_target_times = [];
for st in target_start_times:
unwarped_target_times.append(self.warpTargetTime(st));
unwarped_target_times = np.array(unwarped_target_times);
pim = Image.PlotImage(signal=unwarped_target_times, show_axis=True, xvals=target_start_times,
sampling_rate=sampling_rate, events=self.target_events, xlime=[0, 100], ylims=[0, 110]);
return pim;
def repeatShorterEvents(self, endpoints=False):
n_events = max(len(self.source_events), len(self.target_events));
self.source_events = Event.RepeatToLength(self.source_events, n=n_events, endpoints=endpoints);
self.target_events = Event.RepeatToLength(self.target_events, n=n_events, endpoints=endpoints);
@staticmethod
def FromEvents(source_events, target_events):
w = Warp();
w.source_events = source_events;
w.target_events = target_events;
# w.repeatShorterEvents();
return w
@staticmethod
def LinearInterp(t, a_events, b_events):
n_events = min(len(a_events), len(b_events));
next_a_event_index = n_events;
for s in range(n_events):
if (t < a_events[s].start):
next_a_event_index = s;
break;
prev_a_event_time = 0;
prev_b_event_time = 0;
if (next_a_event_index > 0):
prev_a_event_time = a_events[next_a_event_index - 1].start;
prev_b_event_time = b_events[next_a_event_index - 1].start;
next_a_event_time = a_events[n_events - 1].start;
next_b_event_time = b_events[n_events - 1].start;
if (next_a_event_index < n_events):
next_a_event_time = a_events[next_a_event_index].start;
next_b_event_time = b_events[next_a_event_index].start;
a_event_gap = next_a_event_time - prev_a_event_time;
# b_event_gap = next_b_event_time - prev_b_event_time;
t_progress = t - prev_a_event_time;
# take care of past-the-end case, by simply letting time proceed normally past the last event
if (a_event_gap == 0):
return prev_b_event_time + t_progress;
next_weight = t_progress / (1.0 * a_event_gap);
return (next_weight * next_b_event_time) + ((1.0 - next_weight) * prev_b_event_time);
# additional_points = [];
# for i in range(-10, 11):
# additional_points.append([0.1 * i, 0.51])
@staticmethod
def plotWarpMethodTest(warp_type, additional_points=None, **kwargs):
sev = [];
tev = [];
pts = [[1.0, 1.0],
[-1.0, 1.0],
[0.5, 1.0],
[1.0, 0.5],
[-1.0, 0.5],
[0.5, 0.5],
[1.0, 0.3],
[-1.0, 0.3],
[0.5, 0.3]];
if (additional_points is not None):
pts = pts + additional_points;
currentt = [0.0, 0.0];
sev.append(Event(start=0.0));
tev.append(Event(start=0.0));
for p in pts:
currentt[0] = currentt[0] + p[0];
currentt[1] = currentt[1] + p[1];
sev.append(Event(start=currentt[0]));
tev.append(Event(start=currentt[1]));
# warp_type = 'target_time_source_fraction';
# other_params['acceleration_target_time'] = 0.5;
# other_params['acceleration_source_fraction'] = 0.75;
warpf = Warp.FromEvents(sev, tev);
warpf.setWarpFunc(warp_type, **kwargs);
warpf.plot()
@staticmethod
def SquareInterp(t, a_events, b_events):
n_events = min(len(a_events), len(b_events));
next_a_event_index = n_events;
for s in range(n_events):
if (t < a_events[s].start):
next_a_event_index = s;
break;
prev_a_event_time = 0;
prev_b_event_time = 0;
if (next_a_event_index > 0):
prev_a_event_time = a_events[next_a_event_index - 1].start;
prev_b_event_time = b_events[next_a_event_index - 1].start;
next_a_event_time = a_events[n_events - 1].start;
next_b_event_time = b_events[n_events - 1].start;
if (next_a_event_index < n_events):
next_a_event_time = a_events[next_a_event_index].start;
next_b_event_time = b_events[next_a_event_index].start;
a_event_gap = next_a_event_time - prev_a_event_time;
# b_event_gap = next_b_event_time - prev_b_event_time;
t_progress = t - prev_a_event_time;
# take care of past-the-end case, by simply letting time proceed normally past the last event
if (a_event_gap == 0):
return prev_b_event_time + t_progress;
progress_frac = t_progress / (1.0 * a_event_gap);
next_weight = math.pow(progress_frac, 2);
# accel = 3;
# bweight = math.pow(progress_frac,accel);
# aweight = math.pow(1.0-progress_frac,accel);
# sumweight = aweight+bweight;
# next_weight=bweight/sumweight;
return (next_weight * next_b_event_time) + ((1.0 - next_weight) * prev_b_event_time);
@staticmethod
def GetEventWarpFunc(from_events, to_events, f, lead_time=None, **kwargs):
start_cap_time = min(from_events[0].start, to_events[0].start);
if (lead_time is not None):
start_cap_time = min(start_cap_time, lead_time);
n_events = min(len(from_events), len(to_events));
# f_events = Event.GetUnrolledList(from_events[:n_events], assert_on_folds=True);
f_events = Event.GetUnrolledList(from_events[:n_events]);
t_events = Event.GetUnrolledList(to_events[:n_events]);
def rfunc(t, **kwargs):
next_f_event_index = n_events - 1;
for e in range(n_events):
if (t < f_events[e].unrolled_start):
next_f_event_index = e;
break;
if (next_f_event_index == 0):
from_cap_event = Event(start=f_events[0].start - start_cap_time, weight=0, type='startcap');
to_cap_event = Event(start=t_events[0].start - start_cap_time, weight=0, type='startcap')
return f(t, f_neighbors=[from_cap_event, f_events[0]], t_neighbors=[to_cap_event, t_events[0]]);
else:
return f(t,
f_neighbors=[f_events[next_f_event_index - 1], f_events[next_f_event_index]],
t_neighbors=[t_events[next_f_event_index - 1], t_events[next_f_event_index]]);
return rfunc;
@staticmethod
def WFunc_Quadratic():
def rfunc(t, f_neighbors, t_neighbors, **kwargs):
from_times = Event.ToStartTimes(f_neighbors);
to_times = Event.ToStartTimes(t_neighbors);
from_event_gap = from_times[1] - from_times[0];
t_progress = t - from_times[0];
# avoid divide by 0
if (from_event_gap == 0):
AWARN("Event Gap was 0! Check Warp.py")
return prev_to_event_time + t_progress;
progress_frac = truediv(t_progress, from_event_gap);
next_weight = math.pow(progress_frac, 2);
return (next_weight * to_times[1]) + ((1.0 - next_weight) * to_times[0]);
return rfunc;
@staticmethod
def WFunc_Weight(use_to_weights=None, **kwargs):
print("USING WEIGHT-BASED WARP");
def rfunc(t, f_neighbors, t_neighbors):
from_times = Event.ToStartTimes(f_neighbors);
to_times = Event.ToStartTimes(t_neighbors);
from_event_gap = from_times[1] - from_times[0];
to_event_gap = to_times[1] - to_times[0];
t_progress = t - from_times[0];
# avoid divide by 0
if (from_event_gap == 0):
AWARN("Event Gap was 0! Check Warp.py")
return prev_to_event_time + t_progress;
progress_frac = truediv(t_progress, from_event_gap);
if (use_to_weights):
weight = t_neighbors[1].weight;
else:
weight = f_neighbors[1].weight;
p = 1.0 - weight;
# a = truediv(1.0, (1.0+p*p-2*p)); # if a=b
a = 1.0 - np.power((1.0 - p), 2.0); # b=1
if (progress_frac < p):
next_weight = a * progress_frac;
else:
next_weight = (a * progress_frac) + np.power((progress_frac - p), 2.0);
return (next_weight * to_times[1]) + ((1.0 - next_weight) * to_times[0]);
return rfunc;
@staticmethod
def WFunc_P(p=None, **kwargs):
if (p is None):
p = 0.5;
print(("USING P WARP with P={}".format(p)));
def rfunc(t, f_neighbors, t_neighbors):
from_times = Event.ToStartTimes(f_neighbors);
to_times = Event.ToStartTimes(t_neighbors);
from_event_gap = from_times[1] - from_times[0];
to_event_gap = to_times[1] - to_times[0];
t_progress = t - from_times[0];
# avoid divide by 0
if (from_event_gap == 0):
AWARN("Event Gap was 0! Check Warp.py")
return prev_to_event_time + t_progress;
progress_frac = truediv(t_progress, from_event_gap);
# a = truediv(1.0, (1.0+p*p-2*p)); # if a=b
a = 1.0 - np.power((1.0 - p), 2.0); # b=1
if (progress_frac < p):
next_weight = a * progress_frac;
else:
next_weight = (a * progress_frac) + np.power((progress_frac - p), 2.0);
return (next_weight * to_times[1]) + ((1.0 - next_weight) * to_times[0]);
return rfunc;
@staticmethod
def WFunc_Mouth(p_acceleration_time=0.1, **kwargs):
def rfunc(t, f_neighbors, t_neighbors):
from_times = Event.ToStartTimes(f_neighbors);
to_times = Event.ToStartTimes(t_neighbors);
from_event_gap = from_times[1] - from_times[0];
to_event_gap = to_times[1] - to_times[0];
t_progress = t - from_times[0];
# avoid divide by 0
if (from_event_gap == 0):
AWARN("Event Gap was 0! Check Warp.py")
return prev_to_event_time + t_progress;
progress_frac = truediv(t_progress, from_event_gap);
if (f_neighbors[1].type == 'mouth_open' or t_neighbors[1].type == 'mouth_open'):
p = 1.0 - truediv(p_acceleration_time, to_event_gap);
if (p < 0):
next_weight = math.pow(progress_frac, 2);
else:
# a = truediv(1.0, (1.0+p*p-2*p)); # if a=b
a = 1.0 - np.power((1.0 - p), 2.0); # b=1
if (progress_frac < p):
next_weight = a * progress_frac;
else:
next_weight = (a * progress_frac) + np.power((progress_frac - p), 2.0);
else:
next_weight = progress_frac;
return (next_weight * to_times[1]) + ((1.0 - next_weight) * to_times[0]);
return rfunc;
@staticmethod
def WFunc_targettime_sourcefraction(acceleration_target_time=0.1, acceleration_source_fraction=0.8, **kwargs):
"""
This assumes that you are mapping from the target to the source, as is the most common use case.
:param acceleration_target_time: amount of from time to spend accelerating
:param acceleration_source_fraction: fraction of source to accelerate through
:return:
"""
lin_source_fraction = 1.0 - acceleration_source_fraction;
def rfunc(t, f_neighbors, t_neighbors):
from_times = Event.ToStartTimes(f_neighbors);
to_times = Event.ToStartTimes(t_neighbors);
from_event_gap = from_times[1] - from_times[0];
to_event_gap = to_times[1] - to_times[0];
t_progress = t - from_times[0];
# avoid divide by 0
if (from_event_gap == 0):
AWARN("Event Gap was 0! Check Warp.py")
return to_times[
0] + t_progress; # is this right? doesnt seem to come up, not sure what behavior should be looking back on it...
progress_frac = truediv(t_progress, from_event_gap);
time_left = from_event_gap - t_progress;
p = 1.0 - truediv(acceleration_target_time, from_event_gap);
p2 = p * p;
q = 1.0 - acceleration_source_fraction;
if (acceleration_target_time >= from_event_gap or q > (p2)):
next_weight = math.pow(progress_frac, 2);
elif (time_left >= acceleration_target_time):
lin_t_progress_frac = truediv(t_progress, from_event_gap - acceleration_target_time);
next_weight = lin_t_progress_frac * lin_source_fraction;
else:
pdnom = (p2 - 2.0 * p + 1.0)
# I just used matlab to symbolic inverse matrix of equations, then simplified by hand
a = ((1.0 - q) / pdnom) + (q / (p * (p - 1)));
b = ((2 * p * (q - 1)) / pdnom) - ((q * (p + 1)) / (p2 - p))
# c = ((p2 - (q * (2 * p - 1))) / pdnom) + (q / (p - 1));
c = 1 - a - b;
next_weight = a * progress_frac * progress_frac + b * progress_frac + c;
return (next_weight * to_times[1]) + ((1.0 - next_weight) * to_times[0]);
return rfunc;
@staticmethod
def WFunc_target_source_fractions(acceleration_target_fraction=0.8, acceleration_source_fraction=0.9, **kwargs):
"""
This assumes that you are mapping from the target to the source, as is the most common use case.
:param acceleration_target_fraction: fraction of target to spend accelerating
:param acceleration_source_fraction: fraction of source to accelerate through
:return:
"""
lin_source_fraction = 1.0 - acceleration_source_fraction;
def rfunc(t, f_neighbors, t_neighbors):
print(acceleration_target_fraction)
print(acceleration_source_fraction)
from_times = Event.ToStartTimes(f_neighbors);
to_times = Event.ToStartTimes(t_neighbors);
from_event_gap = from_times[1] - from_times[0];
to_event_gap = to_times[1] - to_times[0];
t_progress = t - from_times[0];
# avoid divide by 0
if (from_event_gap == 0):
AWARN("Event Gap was 0! Check Warp.py")
return to_times[
0] + t_progress; # is this right? doesnt seem to come up, not sure what behavior should be looking back on it...
progress_frac = truediv(t_progress, from_event_gap);
time_left = from_event_gap - t_progress;
p = 1.0 - acceleration_target_fraction;
p2 = p * p;
q = 1.0 - acceleration_source_fraction;
if (acceleration_target_fraction >= 1 or q > (p2)):
next_weight = math.pow(progress_frac, 2);
elif (progress_frac <= p):
lin_t_progress_frac = truediv(t_progress, p * from_event_gap);
next_weight = lin_t_progress_frac * lin_source_fraction;
else:
pdnom = (p2 - 2.0 * p + 1.0)
# I just used matlab to symbolic inverse matrix of equations, then simplified by hand
a = ((1.0 - q) / pdnom) + (q / (p * (p - 1)));
b = ((2 * p * (q - 1)) / pdnom) - ((q * (p + 1)) / (p2 - p))
# c = ((p2 - (q * (2 * p - 1))) / pdnom) + (q / (p - 1));
c = 1 - a - b;
next_weight = a * progress_frac * progress_frac + b * progress_frac + c;
return (next_weight * to_times[1]) + ((1.0 - next_weight) * to_times[0]);
return rfunc;
@staticmethod
def WFunc_AB(const_factor, quad_factor, **kwargs):
def rfunc(t, f_neighbors, t_neighbors):
from_times = Event.ToStartTimes(f_neighbors);
to_times = Event.ToStartTimes(t_neighbors);
from_event_gap = from_times[1] - from_times[0];
t_progress = t - from_times[0];
# avoid divide by 0
if (from_event_gap == 0):
AWARN("Event Gap was 0! Check Warp.py")
return prev_to_event_time + t_progress;
progress_frac = truediv(t_progress, from_event_gap);
next_weight = math.pow(progress_frac, 2);
return (next_weight * to_times[1]) + ((1.0 - next_weight) * to_times[0]);
@staticmethod
def ABWarp():
next_from_event_time = from_events[n_events - 1].start;
next_to_event_time = to_events[n_events - 1].start;
if (next_from_event_index < n_events):
next_from_event_time = from_events[next_from_event_index].start;
next_to_event_time = to_events[next_to_event_index].start;
from_event_gap = next_from_event_time - prev_from_event_time;
# b_event_gap = next_b_event_time - prev_b_event_time;
t_progress = t - prev_from_event_time;
# take care of past-the-end case, by simply letting time proceed normally past the last event
if (from_event_gap == 0):
return prev_to_event_time + t_progress;
progress_frac = t_progress / (1.0 * from_event_gap);
next_weight = math.pow(progress_frac, 2);
return (next_weight * next_to_event_time) + ((1.0 - next_weight) * prev_to_event_time);
@staticmethod
def CubicInterp(t, a_events, b_events):
# def CubicInterp(a_events, b_events, t):
f = Warp.CubicInterpFunc(a_events=a_events, b_events=b_events);
return f(t);
@staticmethod
def LinearInterpFunc(a_events, b_events):
if (a_events[0].start > 0):
ae = np.concatenate((np.asarray([0]), Event.ToStartTimes(a_events)));
else:
ae = Event.ToStartTimes(a_events);
be = Event.ToStartTimes(b_events);
if (len(be) > len(ae)):
be = be[:len(ae)];
elif (len(ae) > len(be)):
ae = ae[:len(be)];
return sp.interpolate.interp1d(ae, be, 'linear', bounds_error=False, fill_value='extrapolate');
@staticmethod
def CubicInterpFunc(a_events, b_events):
event_times = Event.ToStartTimes(a_events);
wevent_times = Event.ToStartTimes(b_events);
minlen = min(len(event_times), len(wevent_times));
event_times = event_times[:minlen];
wevent_times = wevent_times[:minlen];
splinecap = np.arange(0, 1, 0.25);
spline_times = np.concatenate((splinecap + event_times[0] - 1, event_times));
wspline_times = np.concatenate((splinecap + wevent_times[0] - 1, wevent_times));
spline_times = np.append(spline_times, splinecap + 0.5 + spline_times[-1]);
wspline_times = np.append(wspline_times, splinecap + 0.5 + wspline_times[-1]);
# get spline to translate initial times to warped times
splineX = sp.interpolate.interp1d(spline_times, wspline_times, kind='cubic', bounds_error=False,
fill_value='extrapolate');
return splineX;
================================================
FILE: src/video2npz/visbeat3/visbeat3/__init__.py
================================================
from .VisBeatImports import *
from .Video import *
from .VideoClip import *
from .VideoSource import *
from .VisBeatExampleVideo import VisBeatExampleVideo
import re
import os;
import shutil
from .SourceLocationParser import ParseSourseLocation
VISBEAT_ASSETS_DIR = './VisBeatAssets/';
from . import fileui
fileui.INITIAL_DIR = VISBEAT_ASSETS_DIR;
def SetAssetsDir(assets_dir):
global VISBEAT_ASSETS_DIR;
VISBEAT_ASSETS_DIR = assets_dir;
make_sure_dir_exists(assets_dir);
AINFORM("VISBEAT_ASSETS_DIR set to {}".format(VISBEAT_ASSETS_DIR));
make_sure_dir_exists(GetVideoSourcesDir());
temp_dir = os.path.join(VISBEAT_ASSETS_DIR, 'TEMP_FILES'+os.sep);
make_sure_dir_exists(temp_dir);
Video.VIDEO_TEMP_DIR = temp_dir;
fileui.INITIAL_DIR = VISBEAT_ASSETS_DIR;
def GetAssetsDir():
return VISBEAT_ASSETS_DIR;
def GetVideoSourcesDir():
video_sources_dir = os.path.join(GetAssetsDir(), 'VideoSources'+os.sep);
make_sure_dir_exists(video_sources_dir);
return video_sources_dir;
def PullVideo(name=None, source_location=None, max_height=240, **kwargs):
if(isinstance(name, VisBeatExampleVideo)):
assert(source_location is None), 'Provided VisBeatExampleVideo and source location? What are you trying to do?';
source_location = name.url;
vname = name.name;
elif(name is None):
assert(source_location is not None), "Must provide an argument to pullvideo";
sloc = ParseSourseLocation(source_location);
vname =sloc.code;
else:
vname = name;
vs = GetVideoSource(vname);
if(vs and vs.source_location==source_location):
v = vs.getVersion(max_height=max_height);
v.load(features_to_load = 'all');
return v;
print("destination:", GetVideoSourcesDir(), "name:", vname, "source_location:", source_location)
vs = VideoSource.NewVideoSource(destination=GetVideoSourcesDir(), name=vname, source_location=source_location, max_height=max_height, **kwargs);
v = vs.getVersion(max_height=max_height);
return v;
def ClipVideo(video, time_range, max_height=240):
video_fullres = video.source.getVersion();
vclip = video_fullres.VideoClip(start=time_range[0], end=time_range[1]);
vcdir = video.source.getDirForVersion(version_label='{}_{}'.format(str(time_range[0]), str(time_range[1])), version_group='Clips');
make_sure_dir_exists(vcdir);
vcname = video.getName() + 'clip_{}_{}'.format(str(time_range[0]), str(time_range[1]));
vcpath = os.path.join(vcdir, vcname+'.mp4');
vclip.write(output_path=vcpath);
vs = VideoSource.NewVideoSource(destination=GetVideoSourcesDir(), name=vcname, source_location=vcpath);
return vs.getVersion(max_height=max_height);
def GetVideoSource(name):
vname = name;
if (isinstance(name, VisBeatExampleVideo)):
vname = name.name;
path = os.path.join(GetVideoSourcesDir(), vname) + os.sep;
if (os.path.isdir(path)):
return VideoSource(path=path);
def LoadVideo(name, max_height=240):
vname = name;
if (isinstance(name, VisBeatExampleVideo)):
vname = name.name;
path = os.path.join(GetVideoSourcesDir(), vname)+os.sep;
if(os.path.isdir(path)):
vs = VideoSource(path=path);
v = vs.getVersion(max_height=max_height);
v.load(features_to_load = 'all');
return v;
else:
return None;
def Dancefer(source_video, target,
synch_video_beat=0, synch_audio_beat=0,
beat_offset = 0, leadin = None, nbeats=None,
source_harmonic = None, target_harmonic = None, source_harmonic_offset=None, target_harmonic_offset=None,
force_recompute=None, warp_type = 'quad',
name_tag=None, name_tag_prefix=None, output_path = None,
**kwargs):
"""
:param source_video: video to warp
:param target: music to warp to
:param synch_video_beat: integer specifying a beat (as in the nth beat) from the video to synchronize with synch_audio_beat
:param synch_audio_beat: integer specifying a beat (as in the nth beat) from the video to synchronize with synch_video_beat
:param beat_offset: Lets you offset which beats you want to render. This is mostly for testing different parts of an output.
:param leadin: how many beats before the synch beats to render
:param nbeats: lets you restrict output to rendering n beats
:param source_harmonic: can be None, 'half', or 'double'. 'half' will use every other beat, which you can offset with source_harmonic_offset. 'double' will add an additional beat between every consecutive beat. update - added 'third' for waltzes.
:param target_harmonic: can be None, 'half', or 'double'. 'half' will use every other beat, which you can offset with source_harmonic_offset. 'double' will add an additional beat between every consecutive beat. update - added 'third' for waltzes.
:param source_harmonic_offset: optional offset for harmonic
:param target_harmonic_offset: optional offset for harmonic
:param force_recompute:
:param warp_type:
:param name_tag:
:param name_tag_prefix:
:param output_path:
:param kwargs:
:return:
"""
if((output_path is not None) and (not force_recompute)):
if(os.path.exists(output_path)):
return Video(output_path);
if(isinstance(target, Video)):
target_audio = target.getAudio();
else:
target_audio = target;
synchaudio = synch_audio_beat;
synchvideo = synch_video_beat;
lead_in = leadin;
if(lead_in is None):
lead_in = min(synchaudio, synchvideo);
elif(isinstance(lead_in, str) and lead_in[0]=='<'):
# lead_in = min(synchaudio, synchvideo, int(lead_in));
lead_in = min(synchaudio, int(lead_in));
start_audio_beat = synchaudio-lead_in;
start_video_beat = synchvideo-lead_in;
if(beat_offset and beat_offset>0):
start_audio_beat = start_audio_beat+beat_offset;
start_video_beat = start_video_beat+beat_offset;
print(("Warping {} to {}".format(source_video.getName(), target_audio.getName())));
bitrate = None;
vbeats = source_video.audio.getBeatEvents()
tbeats = target_audio.getBeatEvents()
if(start_video_beat < 0):
if(synchvideo == 0):
vbeats = [vbeats[0].clone()]+vbeats;
vbeats[0].start = vbeats[0].start-(vbeats[2].start-vbeats[1].start);
vbadd = Event.SubdivideIntervals(vbeats[:2], -start_video_beat);
vbeats = vbadd+vbeats[2:];
start_video_beat = 0;
vbeats = vbeats[start_video_beat:];
tbeats = tbeats[start_audio_beat:];
if(source_harmonic=='half'):
vbeats = Event.Half(vbeats, source_harmonic_offset);
elif (source_harmonic == 'third'):
vbeats = Event.Third(vbeats, source_harmonic_offset);
elif(source_harmonic == 'double'):
vbeats = Event.Double(vbeats);
if (target_harmonic == 'half'):
tbeats = Event.Half(tbeats, target_harmonic_offset);
elif (target_harmonic == 'third'):
tbeats = Event.Third(tbeats, target_harmonic_offset);
elif (target_harmonic == 'double'):
tbeats = Event.Double(tbeats);
if(nbeats):
print(("Rendering {} beats of result".format(nbeats)))
if(len(vbeats)>nbeats):
vbeats = vbeats[:nbeats];
print((len(vbeats)))
if(len(tbeats)>nbeats):
tbeats = tbeats[:nbeats];
print((len(tbeats)))
else:
if(vbeats[-1].start0):
start_audio_beat = start_audio_beat+beat_offset;
start_video_beat = start_video_beat+beat_offset;
print(("Warping {} to {}".format(source_video.getName(), target_audio.getName())));
bitrate = None;
vbeats = source_beats;
if(source_beats is None):
vbeats = source_video.getVisualBeats();
tbeats = target_beats;
if(target_beats is None):
tbeats = target_audio.getBeatEvents();
if(start_video_beat < 0):
if(synchvideo == 0):
vbeats = [vbeats[0].clone()]+vbeats;
vbeats[0].start = vbeats[0].start-(vbeats[2].start-vbeats[1].start);
vbadd = Event.SubdivideIntervals(vbeats[:2], -start_video_beat);
vbeats = vbadd+vbeats[2:];
start_video_beat = 0;
vbeats = vbeats[start_video_beat:];
tbeats = tbeats[start_audio_beat:];
if (source_harmonic == 'half'):
vbeats = Event.Half(vbeats, source_harmonic_offset);
elif (source_harmonic == 'third'):
vbeats = Event.Third(vbeats, source_harmonic_offset);
elif (source_harmonic == 'double'):
vbeats = Event.Double(vbeats);
if (target_harmonic == 'half'):
tbeats = Event.Half(tbeats, target_harmonic_offset);
elif (target_harmonic == 'third'):
tbeats = Event.Third(tbeats, target_harmonic_offset);
elif (target_harmonic == 'double'):
tbeats = Event.Double(tbeats);
if(nbeats):
print(("Rendering {} beats of result".format(nbeats)))
if(len(vbeats)>nbeats):
vbeats = vbeats[:nbeats];
print((len(vbeats)))
if(unfold_to_n):
vbeats = Event.UnfoldToN(vbeats, unfold_to_n, momentum=momentum);
if (len(tbeats) > len(vbeats)):
tbeats = tbeats[:len(vbeats)];
if(warp_type is 'weight'):
vbeats = source_video.visualBeatsFromEvents(vbeats);
if(name_tag is None):
name_tag = warp_type+'_sab_'+str(start_audio_beat)+'_svb_'+str(start_video_beat);
if(name_tag_prefix is not None):
name_tag = name_tag+name_tag_prefix;
warp_args = dict(target=target_audio,
source_events=vbeats,
target_events = tbeats,
warp_type=warp_type,
force_recompute=force_recompute,
name_tag = name_tag)
if(bitrate):
warp_args.update(dict(bitrate=bitrate));
warp_args.update(kwargs);
warped_result = source_video.getWarped(**warp_args);
if(output_path):
final_output_path = output_path;
if(os.path.isfile(final_output_path)):
output_filename = os.path.basename(output_path);
name_parts = os.path.splitext(output_filename);
output_filename_base = name_parts[0];
output_directory_path = os.path.dirname(output_path);
if (output_directory_path == ''):
output_directory_path = '.'
output_ext = name_parts[1];
ntry = 1;
tryname = output_filename_base+ '_' + str(ntry);
while (os.path.isfile(os.path.join(output_directory_path, tryname+output_ext)) and ntry<100):
ntry = ntry+1;
tryname = output_filename_base + '_' + str(ntry);
final_output_path = os.path.join(output_directory_path, tryname + output_ext);
shutil.copy2(src=warped_result.getPath(), dst=final_output_path);
n_frames_total = warped_result.num_frames_total;
warp_used = warped_result.getInfo('warp_used');
warped_result_final = Video(path = final_output_path, num_frames_total=n_frames_total);
warped_result_final.setInfo(label='warp_used', value=warp_used);
os.remove(warped_result.getPath())
warped_result = warped_result_final;
return warped_result;
def getVBSegments(self,source_video,
source_beats = None,
search_tempo=None,
search_window=None,
max_height=None,
beat_limit=None,
n_return=None,
unary_weight=None,
binary_weight=None,
break_on_cuts=None,
peak_vars=None):
source = source_video;
if(source_beats is None):
if (peak_vars is not None):
vbeats = source.getFeature('simple_visual_beats', **peak_vars);
else:
vbeats = source.getFeature('simple_visual_beats');
else:
vbeats = source_beats;
#
if (search_tempo is not None):
tempo = search_tempo;
beat_time = np.true_divide(60.0, tempo);
clips = VisualBeat.PullOptimalPaths_Basic(vbeats, target_period=beat_time, unary_weight=unary_weight,
binary_weight=binary_weight, break_on_cuts=break_on_cuts,
window_size=search_window);
else:
clips = VisualBeat.PullOptimalPaths_Autocor(vbeats, unary_weight=unary_weight, binary_weight=binary_weight,
break_on_cuts=break_on_cuts, window_size=search_window);
if (beat_limit is None):
beat_limit = 2;
print(("There were {} candidates".format(len(vbeats))));
nclips = 0;
segments = [];
for S in clips:
if (len(S) > beat_limit):
nclips = nclips + 1;
segments.append(S);
if (n_return is not None):
segments.sort(key=len, reverse=True);
segments = segments[:n_return];
return segments;
================================================
FILE: src/video2npz/visbeat3/visbeat3/_dancefer_examples.py
================================================
from .VisBeatExampleVideo import *
from ._music_examples import *
# dances = [];
dances.append(VisBeatExampleVideo(display_name = 'SNSD: HAHAHA', name='snsd_hahaha', url='https://www.youtube.com/watch?v=OxNsx0eMsGU', start_beat=42));
dances.append(VisBeatExampleVideo(display_name = 'Boston Dynamics: Spot the Dancing Robot', name='spot_the_dancing_robot', url='https://www.youtube.com/watch?v=kHBcVlqpvZ8', start_beat = 9));
dances.append(VisBeatExampleVideo(display_name = 'Michael Jackson: Thriller', name = 'thriller', url = 'https://www.youtube.com/watch?v=wlHUJbl-t7o', start_beat = 4))
dances.append(VisBeatExampleVideo(display_name = 'LMFAO: Party Rock Anthem',name='party_rock', url='https://www.youtube.com/watch?v=WZRMpC_wh0M', start_beat=4, leadin=4))
dances.append(VisBeatExampleVideo(display_name = 'BTS: Boy in Luv', name='boy_in_luv', url='https://www.youtube.com/watch?v=gqz6Adx63w8', start_beat=34));
dances.append(VisBeatExampleVideo(display_name = 'Thays Monaro: Redneck Woman',name='gwilson_redneck_woman', url='https://www.youtube.com/watch?v=1apy8YPygKo', start_beat=65))
dances.append(VisBeatExampleVideo(display_name = 'Music Express: Dancing in the Street', name='dance_cc_dancing_in_the_street', url='https://www.youtube.com/watch?v=fzZ2CoL3IMQ', start_beat=3));
dances.append(VisBeatExampleVideo(display_name = 'Barney and Friends: Mr. Golden Sun',name='barney_mr_golden_sun', url='https://www.youtube.com/watch?v=ya4yyg9XiI4', start_beat=24))
dances.append(VisBeatExampleVideo(display_name = 'M. Express: Friends Forever', name='dance_cc_friends_forever', url='https://www.youtube.com/watch?v=T6ZX6wkJ5Ns', start_beat=2));
dances.append(VisBeatExampleVideo(display_name = 'Just Dance Kids: Gummy Bear',name='jdk_gummy_bear', url='https://www.youtube.com/watch?v=KVE-T2_vLpY', start_beat=60))
dances.append(VisBeatExampleVideo(display_name = 'M. Express: Feelin Good',name='dance_cc_feelin_good', url='https://www.youtube.com/watch?v=uBWp9rr6w08', start_beat=49));
# dances.append(VisBeatExampleVideo(display_name = 'Kiesza: Hideaway', name='hideaway', url='https://www.youtube.com/watch?v=Vnoz5uBEWOA', start_beat=124))
dances.append(VisBeatExampleVideo(display_name = 'SNSD: Genie', name='snsd_genie', url='https://www.youtube.com/watch?v=6SwiSpudKWI', start_beat=141));
dances.append(VisBeatExampleVideo(display_name = 'Momoland Boom Boom', name='momoland_boom_boom', url='https://www.youtube.com/watch?v=0HKfjsM2hSw', start_beat = 145));#33#
dances.append(VisBeatExampleVideo(display_name = 'Snap: Rhythm is a Dancer', name='rhythm_is_a_dancer', url='https://www.youtube.com/watch?v=fDWFVI8PQOI', start_beat=65));
dances.append(VisBeatExampleVideo(display_name = 'Janet Jackson: Rhythm Nation', name='rhythm_nation', url='https://www.youtube.com/watch?v=OAwaNWGLM0c', start_beat = 226));
dances.append(VisBeatExampleVideo(display_name = 'BTS Just One Day', name='bts_just_one_day', url='https://www.youtube.com/watch?v=KQeAg45p2UI', start_beat=200));
# dances.append(VisBeatExampleVideo(display_name = '', name='', url='', start_beat = ));
# dances.append(VisBeatExampleVideo(display_name = '', name='', url='', start_beat = ));
# dances.append(VisBeatExampleVideo(display_name = '', name='', url='', start_beat = ));
# dances.append(VisBeatExampleVideo(display_name = '', name='', url='', start_beat = ));
# dances.append(VisBeatExampleVideo(display_name = '', name='', url='', start_beat = ));
================================================
FILE: src/video2npz/visbeat3/visbeat3/_dancify_examples.py
================================================
from .VisBeatExampleVideo import *
from ._music_examples import *
dances.append(VisBeatExampleVideo(display_name = 'Turtle', name='turtle', url='https://www.youtube.com/watch?v=PWD4gktEUAY', start_beat=0));
dances.append(VisBeatExampleVideo(display_name = 'Louie The Cat', name='louie_the_cat', url='https://www.youtube.com/watch?v=hqFG6d86ygI', start_beat=None));
accidental_dances = [];
accidental_dances.append(VisBeatExampleVideo(display_name="Trump IACP Full", name="trump_iacp_full", url='https://www.youtube.com/watch?v=pu4dPTi6SEg', target_n_beats = 7))
accidental_dances.append(VisBeatExampleVideo(display_name="Trump Tax Reform WH.gov Full", name="trump_taxreform_remarks_1", url="https://www.youtube.com/watch?v=zB-lhTEQdKY"))
# accidental_dances.append(VisBeatExampleVideo(display_name="", name="", url=""))
synth = [];
synth.append(VisBeatExampleVideo(name='synth_ball_metronome_w_noise', url = 'https://youtu.be/-mqJIGdro6A', start_beat = None));
synth.append(VisBeatExampleVideo(name='synth_test_ball_1', url = 'https://youtu.be/SjXsMYBJEF8', start_beat = None));
synth.append(VisBeatExampleVideo(name='synth_test_ball_1_gt_impact_sounds', url = 'https://youtu.be/tsdsQm4iTNg', start_beat = None));
================================================
FILE: src/video2npz/visbeat3/visbeat3/_mediafiles.py
================================================
import os
VB_MEDIA_UTILS_PATH = os.path.abspath(__file__)
VB_MEDIA_UTILS_DIR = os.path.abspath(os.path.dirname(__file__));
MEDIAFILES_DIR = os.path.join(VB_MEDIA_UTILS_DIR, 'assets'+os.sep)
AUDIO_FILES_DIR = os.path.join(MEDIAFILES_DIR, 'audio'+os.sep);
AUDIO_FILES = [];
AUDIO_FILE_PATHS = {};
for filename in os.listdir(AUDIO_FILES_DIR):
# if(reduce(lambda x,y: x or y, map(lambda ext: filename.lower().endswith(ext), Audio.MEDIA_FILE_EXTENSIONS()))):
AUDIO_FILES.append(filename);
AUDIO_FILE_PATHS[filename]=(os.path.join(AUDIO_FILES_DIR, filename));
def GetTestAudioPath(filename):
return AUDIO_FILE_PATHS[filename];
VIDEO_FILES_DIR = os.path.join(MEDIAFILES_DIR, 'video'+os.sep);
VIDEO_FILES = [];
VIDEO_FILE_PATHS = [];
if(os.path.exists(VIDEO_FILES_DIR)):
for filename in os.listdir(VIDEO_FILES_DIR):
VIDEO_FILES.append(filename);
VIDEO_FILE_PATHS.append(os.path.join(VIDEO_FILES_DIR, filename));
IMAGE_FILES_DIR = os.path.join(MEDIAFILES_DIR, 'images'+os.sep);
IMAGE_FILES = [];
IMAGE_FILE_PATHS = {};
for filename in os.listdir(IMAGE_FILES_DIR):
IMAGE_FILES.append(filename);
IMAGE_FILE_PATHS[filename] = (os.path.join(IMAGE_FILES_DIR, filename));
def GetTestImagePath(filename=None):
if(filename is None):
filename = "VisBeatWatermark.png"
return IMAGE_FILE_PATHS[filename];
def GetVBMarkPath():
return IMAGE_FILE_PATHS["VisBeatWatermark.png"];
================================================
FILE: src/video2npz/visbeat3/visbeat3/_music_examples.py
================================================
from .VisBeatExampleVideo import *
dances = [];
music = [];
music.append(VisBeatExampleVideo(display_name = 'U Cant Touch This', name='cant_touch_this', url='https://www.youtube.com/watch?v=_NNYI8VbFyY', start_beat=2, leadin=2));
music.append(VisBeatExampleVideo(display_name = 'Mary Poppins: Supercalifrag.', name='supercalifrag', url='https://www.youtube.com/watch?v=rihNRTTcztQ', start_beat = 5, leadin = 3));
music.append(VisBeatExampleVideo(display_name = 'Migos: Walk It Talk It',name='walkit_talkit', url='https://www.youtube.com/watch?v=fGqdIPer-ms', start_beat = 134));
music.append(VisBeatExampleVideo(display_name = 'Rammstein: Du Hast',name='du_hast', url = 'https://www.youtube.com/watch?v=W3q8Od5qJio', start_beat = 193));
music.append(VisBeatExampleVideo(display_name = 'Taylor Swift: Shake It Off', name='shake_it_off', url='https://www.youtube.com/watch?v=nfWlot6h_JM', start_beat=112));
music.append(VisBeatExampleVideo(display_name = 'Michael Jackson: Billie Jean', name='billie_jean', url='https://www.youtube.com/watch?v=Zi_XLOBDo_Y', start_beat=169));
music.append(VisBeatExampleVideo(display_name = 'Hall and Oates: You Make My Dreams', name='you_make_my_dreams', url='https://www.youtube.com/watch?v=EErSKhC0CZs', start_beat=95));
music.append(VisBeatExampleVideo(display_name = 'Parry Gripp: Breakfast Burrito',name="breakfast_burrito", url='https://www.youtube.com/watch?v=prPjpwsGiws', start_beat=32))
music.append(VisBeatExampleVideo(display_name = 'Pharrell Williams: Happy',name='happy', url='https://www.youtube.com/watch?v=y6Sxv-sUYtM', start_beat=69, leadin=3))
music.append(VisBeatExampleVideo(display_name = 'Aqua: Barbie Girl',name='barbie_girl', url='https://www.youtube.com/watch?v=ZyhrYis509A', start_beat=67, leadin=4));
music.append(VisBeatExampleVideo(display_name = 'Jhameel: Montage',name='music_cc_jhameel_montage', url='https://www.youtube.com/watch?v=vYr_nCmsgoE', start_beat=103))
music.append(VisBeatExampleVideo(display_name = 'Vengaboys: We like to Party',name='we_like_to_party', url='https://www.youtube.com/watch?v=6Zbi0XmGtMw', start_beat=65))
music.append(VisBeatExampleVideo(display_name = 'Rick Astley: Never Gonna Give You Up', name='rick_roll', url='https://www.youtube.com/watch?v=dQw4w9WgXcQ', start_beat = 3, leadin=3));
music.append(VisBeatExampleVideo(display_name = 'Mark Ronson: Uptown Funk ft. Bruno Mars', name='uptown_funk', url='https://www.youtube.com/watch?v=OPf0YbXqDm0', start_beat=32));
music.append(VisBeatExampleVideo(display_name = 'Missy Elliot: Work It', name='work_it', url='https://www.youtube.com/watch?v=cjIvu7e6Wq8', start_beat=52, leadin = 1));
music.append(VisBeatExampleVideo(display_name = 'If I only had a brain', name='only_had_a_brain', url='https://www.youtube.com/watch?v=nauLgZISozs', start_beat=14));
music.append(VisBeatExampleVideo(display_name = 'Kane Brown: Lose It', name='kb_lose_it', url='https://www.youtube.com/watch?v=Z8nkc2KKjd8', start_beat=56));
# music.append(VisBeatExampleVideo(display_name = 'Check Meowt', name='pg_check_meowt', url='https://youtu.be/NjKYX4bQpf0', start_beat = 16));
# music.append(VisBeatExampleVideo(display_name = 'Wu-Tang Clan: Protect Ya Neck', name='wutang_protect_ya_neck', url='https://www.youtube.com/watch?v=R0IUR4gkPIE', start_beat = None));
# music.append(VisBeatExampleVideo(display_name = 'Ice Cube: It Was A Good Day',name='it_was_a_good_day', url='https://www.youtube.com/watch?v=h4UqMyldS7Q', start_beat=64, harmonic = 'half'));
================================================
FILE: src/video2npz/visbeat3/visbeat3/command_line.py
================================================
================================================
FILE: src/video2npz/visbeat3/visbeat3/fileui/__init__.py
================================================
from sys import platform
if platform == "linux" or platform == "linux2":
PLATFORM = 'linux';
elif platform == "darwin":
PLATFORM = 'osx';
elif platform == "win32":
PLATFORM = 'windows'
SUPPORTED = False;
INITIAL_DIR ='./';
if(PLATFORM == 'osx'):
from . import uipath
def GetFilePath(initial_path=None):
if(initial_path is None):
return uipath.uiGetFilePath(initial_path=INITIAL_DIR);
else:
return uipath.uiGetFilePath(initial_path);
def GetDirectory(initial_path=None):
if(initial_path is None):
return uipath.uiGetDirectory(initial_path=INITIAL_DIR);
else:
return uipath.uiGetDirectory(initial_path);
def GetSaveFilePath(initial_path=None, file_extension = None):
if(initial_path is None):
return uipath.uiGetSaveFilePath(initial_path=INITIAL_DIR, file_extension=file_extension);
else:
return uipath.uiGetSaveFilePath(initial_path=initial_path, file_extension=file_extension);
def Show(path):
uipath.showInFinder(path);
def Open(path):
uipath.openOSX(path);
SUPPORTED = True;
else:
def GetFilePath(initial_path=None):
return None;
def GetDirectory(initial_path=None):
return None;
def GetSaveFilePath(initial_path=None, file_extension = None):
return None;
def Show(path):
return None;
def Open(path):
return None;
================================================
FILE: src/video2npz/visbeat3/visbeat3/fileui/uipath.py
================================================
import os
import subprocess
def uiGetFilePath(initial_path=None):
try:
if(initial_path):
output = subprocess.check_output("osascript -e 'set strPath to POSIX file \"{}\"' -e 'set theDocument to choose file with prompt \"Please select a document to process:\" default location strPath' -e 'set theDocument to (the POSIX path of theDocument)'".format(initial_path), shell=True)
else:
output = subprocess.check_output("osascript -e 'set theDocument to choose file with prompt \"Please select a document to process:\"' -e 'set theDocument to (the POSIX path of theDocument)'", shell=True)
return output.replace('\n', '');
except subprocess.CalledProcessError as e:
print((e.output));
# assert(False)
# grabpath = get_ipython().run_cell_magic(u'bash', u'', "osascript -e 'set theDocument to choose file with prompt \"Please select a document to process:\"' -e 'set theDocument to (the POSIX path of theDocument)'>&2")
def uiGetDirectory(initial_path=None):
try:
if(initial_path):
output = subprocess.check_output("osascript -e 'set strPath to POSIX file \"{}\"' -e 'set thedir to choose folder with prompt \"Please select a file:\" default location strPath' -e 'set thedir to (the POSIX path of thedir)'".format(initial_path), shell=True)
else:
output = subprocess.check_output("osascript -e 'set thedir to choose folder with prompt \"Please select a directory:\"' -e 'set thedir to (the POSIX path of thedir)'", shell=True)
return output.replace('\n', '');
except subprocess.CalledProcessError as e:
print((e.output));
# assert(False)
def uiGetSaveFilePath(initial_path=None, file_extension=None):
try:
osastr = "osascript ";
if(initial_path):
osastr = osastr+"-e 'set strPath to POSIX file \"{}\"' ".format(initial_path);
osastr = osastr+"-e 'set theDocument to choose file name with prompt \"Save As File:\" ";
if(initial_path):
osastr = osastr+"default location strPath";
osastr = osastr+"' ";
osastr = osastr+"-e 'set theDocument to (the POSIX path of theDocument)'"
output = subprocess.check_output(osastr, shell=True);
ostring = output.replace('\n', '');
if (file_extension is not None):
if (not ostring.endswith(file_extension)):
ostring = ostring + file_extension;
return ostring;
except subprocess.CalledProcessError as e:
AWARN('ERROR')
print((e.output));
def showInFinder(path):
return openOSX(get_dir_from_path(path));
def openOSX(path):
return subprocess.check_output("open {}".format(put_string_in_quotes(path)), shell=True);
def put_string_in_quotes(s):
return "\""+s+"\""
def get_file_name_from_path(pth):
return os.path.split(pth)[1];
def get_dir_from_path(pth):
return (os.path.split(pth)[0]+os.sep);
================================================
FILE: src/video2npz/visbeat3/visbeat3/vbgui/BeatGUI.py
================================================
from visbeat3.AImports import *
VIEWER_INSTALLED = 1;
try:
import vbwidget as Viewer
except ImportError as e:
VIEWER_INSTALLED = 0;
AWARN("VBViewer not installed. Consider installing for full functionality.")
from ..TimeSignal import *
from ..EventList import *
#this is what media should call to get its gui object
def media_GUI_func(self):
if (self._gui is None):
self._gui = BeatGUI();
self._gui.media = self;
return self._gui;
class BeatGUI(AObject):
"""
"""
def AOBJECT_TYPE(self):
return 'BeatGUI';
def __init__(self, media=None, path=None, clear_temp=None):
"""If you provide a directory, it will look for a existing AFileManager.json in that directory, or create one if it does not already exist.
If you provide a json, it will use that json, unless the json doesn't exist, in which case it will complain...
"""
AObject.__init__(self, path=path);
if(media is not None):
self.media = media;
def initializeBlank(self):
AObject.initializeBlank(self);
self._widget = None;
self._media = None;
def getJSONName(self):
return self.AOBJECT_TYPE()+".json";
#########
#
@property
def media(self):
return self._getMedia();
def _getMedia(self):
return self._media;
@media.setter
def media(self, value):
self._setMedia(value);
def _setMedia(self, value):
self._media = value;
#
#
@property
def media_type(self):
return self._getMediaType();
def _getMediaType(self):
if(self.media is None):
return None;
else:
return self.media.AObjectType();
#
#
@property
def widget(self):
return self._getWidget();
def _getWidget(self):
if (self._widget is None):
self._widget = Viewer.VBVSignal();
return self._widget;
@widget.setter
def widget(self, value):
self._setWidget(value);
def _setWidget(self, value):
self._widget = value;
#
#
@property
def frame_rate(self):
return self._getFrameRate();
def _getFrameRate(self):
gfr = self.widget.frame_rate;
if (gfr is None):
media = self.media;
if (media is not None):
gfr = media.getFrameRate();
return gfr;
@frame_rate.setter
def frame_rate(self, value):
self._setFrameRate(value);
def _setFrameRate(self, value):
self.widget.frame_rate = float(value);
#
#
@property
def frame_offset(self):
return self._getFrameOffset();
def _getFrameOffset(self):
return self.widget.frame_offset;
@frame_offset.setter
def frame_offset(self, value):
self._setFrameOffset(value);
def _setFrameOffset(self, value):
self.widget.frame_offset = value;
#
def run(self, local_saliency=None, frame_rate = None, eventlist = 'default', frame_offset=None):
if(frame_rate is None):
# self.widget.frame_rate = float(self.getMedia().getFrameRate());
self.frame_rate = self.media._getFrameRate();
else:
# self.widget.frame_rate = float(frame_rate);
self.frame_rate = frame_rate;
if(local_saliency is None):
self.widget.signal = self.media.getLocalRhythmicSaliency().tolist();
# self.widget.signal = self.getBothWayVisualImpactEnvelope(highpass_window_seconds=None, force_recompute = True).tolist();
else:
self.widget.signal = local_saliency.tolist();
if(frame_offset is None):
self.frame_offset = 0;
elif(frame_offset is 'guess'):
self.frame_offset = self.guessFrameOffset();
else:
self.frame_offset = frame_offset;
if(eventlist is None):
self.widget.events = [];
elif(eventlist == 'default'):
self.widget.events = EventList._toGUIDicts(self.media.getEventList());
else:
self.widget.events = EventList._toGUIDicts(eventlist);
self.widget.data_string = self.media.getStringForHTMLStreamingBase64();
return self.widget;
def guessFrameOffset(self):
if(isinstance(self.media, Video)):
return self.media.reader.get_length() - self.media.n_frames();
else:
return 0;
def deactivateAllEvents(self):
newes = []
gevents = self.getEventDicts();
for e in gevents:
newe = e;
newe['is_active']=0;
newes.append(newe);
self.widget.events = [];
self.widget.events = newes;
def activateAllEvents(self):
newes = []
gevents = self.getEventDicts();
for e in gevents:
newe = e;
newe['is_active'] = 1;
newes.append(newe);
self.widget.events = [];
self.widget.events = newes;
def activatePattern(self, pattern=None, prefix=None, apply_to_active=None):
assert(pattern), "must provide pattern to activate"
newes = []
gevents = self.getGUIEventDicts();
counter = 0;
prefix_length = 0;
if(prefix_length is not None):
prefix_length = len(prefix);
for i, e in enumerate(gevents):
if (apply_to_active):
if (e.get('is_active')):
if (counter < prefix_length):
e['is_active']=prefix[counter];
else:
e['is_active'] = pattern[(counter - prefix_length) % len(pattern)];
counter = counter + 1;
else:
print(("Skipping beat {}, inactive".format(i)));
else:
if (i < prefix_length):
e['is_active'] = prefix[i];
else:
e['is_active'] = pattern[(i - prefix_length) % len(pattern)];
newes.append(e);
self.widget.events = [];
self.widget.events = newes;
def shiftEventsByNFrames(self, n_frames=None):
assert(n_frames), "must provide number of frames to shift by"
newes = []
gevents = self.getEventDicts();
sample_step = np.true_divide(1.0,self.getFrameRate());
for e in gevents:
newe = e;
newe['start'] = newe['start']+n_frames*sample_step;
newes.append(newe);
self.widget.events = [];
self.widget.events = newes;
def getActiveEventTimes(self):
gevents = self.getEventDicts(active_only=True);
revents = []
for e in gevents:
revents.append(e.get('time'));
return np.asarray(revents);
def getEventTimes(self):
gevents = self.getEventDicts();
revents = []
for e in gevents:
revents.append(e.t);
return np.asarray(revents);
def getEvents(self, active_only=None):
return Event._FromGUIDicts(self.getEventDicts(active_only = active_only));
def getEventList(self, active_only=None):
elist = EventList._FromGUIDicts(self.getEventDicts(active_only=active_only));
elist.setInfo(label='html_frame_offset', value=self.getFrameOffset());
return elist;
def getActiveEvents(self):
return self.getEvents(active_only=True);
def getEventDicts(self, active_only = None):
gevents = self.widget.events[:];
if(not active_only):
return gevents;
else:
nevents = []
for e in gevents:
if(e.get('is_active')):
nevents.append(e);
return nevents;
def saveEvents(self, save_path = None):
elist = self.getEventList(active_only=False);
if(save_path is not None):
elist.writeToJSON(json_path=save_path);
self.widget.last_save_path = save_path;
else:
save_path = self.widget.last_save_path;
if(save_path is None):
save_path = uiGetSaveFilePath(file_extension='.json');
if(save_path is not None):
elist.writeToJSON(json_path=save_path);
self.widget.last_save_path = save_path;
def saveEventsAs(self, save_path = None):
elist = self.getEventList(active_only=False);
if (save_path is not None):
elist.writeToJSON(json_path=save_path);
self.widget.last_save_path = save_path;
print(('savepath not none {}'.format(save_path)))
else:
save_path = uiGetSaveFilePath(file_extension='.json');
print(('savepath from ui {}'.format(save_path)))
if (save_path is not None):
print(('save path from ui {}'.format(save_path)));
elist.writeToJSON(json_path=save_path);
self.widget.last_save_path = save_path;
print(save_path)
def setEvents(self, events):
self.widget.events = Event._ToGUIDicts(events);
def setEventList(self, event_list):
if(event_list.getInfo('html_frame_offset') is not None):
self.widget.frame_offset = event_list.getInfo('html_frame_offset');
self.widget.events = event_list._toGUIDicts();
def loadEvents(self, load_path = None):
if(load_path is None):
load_path = uiGetFilePath();
elist = EventList();
elist.loadFromJSON(json_path=load_path);
self.setEventList(event_list = elist);
def getEventListWithSelectedSegments(self):
eventlist = self.getEventList();
events = eventlist.events;
segments = [];
for i, e in enumerate(events):
if(e.direction>-1): # meaning not a back beat
newseg = [];
for si in range(i, len(events)):
newseg.append(si);
if(events[si].direction<0): # meaning a back beat
break;
segments.append(newseg);
eventlist.setInfo(label='selected_segments', value=segments);
return eventlist;
================================================
FILE: src/video2npz/visbeat3/visbeat3/vbgui/__init__.py
================================================
================================================
FILE: src/video2npz/visbeat3/visbeat3.egg-info/PKG-INFO
================================================
Metadata-Version: 2.1
Name: visbeat3
Version: 0.0.8
Summary: Python3 Implementation for 'Visual Rhythm and Beat' SIGGRAPH 2018
Home-page: https://github.com/haofanwang/visbeat3
Author: Haofan Wang
Author-email: haofanwang.ai@gmail.com
License: UNKNOWN
Description: # visbeat3
This is a migration of [visbeat](http://abedavis.com/visualbeat/) from Python2 to Python3. All credits belong to the original author.
Note: This repo is under development, for any issue, please PR directly!
## Install
```
pip3 install visbeat3
```
or
```
pip3 install -e git+https://github.com/haofanwang/visbeat3.git#egg=visbeat3
```
## Usage
```
import visbeat3 as vb
```
## Reference
```
@inproceedings{davis2018visual,
title={Visual rhythm and beat},
author={Davis, Abe and Agrawala, Maneesh},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops},
pages={2532--2535},
year={2018}
}
```
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Description-Content-Type: text/markdown
================================================
FILE: src/video2npz/visbeat3/visbeat3.egg-info/SOURCES.txt
================================================
.DS_Store
LICENSE
LICENSE.pdf
MANIFEST.in
README.md
setup.cfg
setup.py
test.py
.git/.DS_Store
.git/HEAD
.git/config
.git/description
.git/index
.git/packed-refs
.git/hooks/applypatch-msg.sample
.git/hooks/commit-msg.sample
.git/hooks/fsmonitor-watchman.sample
.git/hooks/post-update.sample
.git/hooks/pre-applypatch.sample
.git/hooks/pre-commit.sample
.git/hooks/pre-merge-commit.sample
.git/hooks/pre-push.sample
.git/hooks/pre-rebase.sample
.git/hooks/pre-receive.sample
.git/hooks/prepare-commit-msg.sample
.git/hooks/push-to-checkout.sample
.git/hooks/update.sample
.git/info/exclude
.git/logs/HEAD
.git/logs/refs/heads/main
.git/logs/refs/remotes/origin/HEAD
.git/objects/pack/pack-390cb819f9aa179975f055403fc70bc5f8d17684.idx
.git/objects/pack/pack-390cb819f9aa179975f055403fc70bc5f8d17684.pack
.git/refs/heads/main
.git/refs/remotes/origin/HEAD
VisBeatAssets/.DS_Store
VisBeatAssets/VideoSources/.DS_Store/VideoSource.json
VisBeatAssets/VideoSources/.DS_Store/Versions/Source/Full/.DS_Store
VisBeatAssets/VideoSources/animation.mp4/.DS_Store
VisBeatAssets/VideoSources/animation.mp4/VideoSource.json
VisBeatAssets/VideoSources/animation.mp4/Data/.DS_Store
VisBeatAssets/VideoSources/animation.mp4/Data/Backups/VideoSource.json
VisBeatAssets/VideoSources/animation.mp4/Data/Features/video/directogram_powers/animation_maxheight_360.pkl
VisBeatAssets/VideoSources/animation.mp4/Versions/.DS_Store
VisBeatAssets/VideoSources/animation.mp4/Versions/Original/.DS_Store
VisBeatAssets/VideoSources/animation.mp4/Versions/Original/maxheight_360/animation.mp4
VisBeatAssets/VideoSources/animation.mp4/Versions/Source/Full/animation.mp4
VisBeatAssets/VideoSources/demo-new.mp4/VideoSource.json
VisBeatAssets/VideoSources/demo-new.mp4/Data/Backups/VideoSource.json
VisBeatAssets/VideoSources/demo-new.mp4/Data/Features/video/directogram_powers/demo-new_maxheight_360.pkl
VisBeatAssets/VideoSources/demo-new.mp4/Versions/Original/maxheight_360/demo-new.mp4
VisBeatAssets/VideoSources/demo-new.mp4/Versions/Source/Full/demo-new.mp4
VisBeatAssets/VideoSources/final_640.mp4/VideoSource.json
VisBeatAssets/VideoSources/final_640.mp4/Data/Backups/VideoSource.json
VisBeatAssets/VideoSources/final_640.mp4/Data/Features/video/directogram_powers/final_640_maxheight_360.pkl
VisBeatAssets/VideoSources/final_640.mp4/Versions/Original/maxheight_360/final_640.mp4
VisBeatAssets/VideoSources/final_640.mp4/Versions/Source/Full/final_640.mp4
VisBeatAssets/VideoSources/wzk1_0.mp4/VideoSource.json
VisBeatAssets/VideoSources/wzk1_0.mp4/Data/Backups/VideoSource.json
VisBeatAssets/VideoSources/wzk1_0.mp4/Data/Features/video/directogram_powers/wzk1_0_maxheight_360.pkl
VisBeatAssets/VideoSources/wzk1_0.mp4/Versions/Original/maxheight_360/wzk1_0.mp4
VisBeatAssets/VideoSources/wzk1_0.mp4/Versions/Source/Full/wzk1_0.mp4
VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/.DS_Store
VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/VideoSource.json
VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238.mp4/.DS_Store
VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238.mp4/VideoSource.json
VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238.mp4/Data/.DS_Store
VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238.mp4/Versions/.DS_Store
VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Data/.DS_Store
VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Data/Backups/VideoSource.json
VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Data/Features/video/directogram_powers/wzk_vlog_beat_enhance1_track1238_maxheight_360.pkl
VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Versions/.DS_Store
VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Versions/Original/maxheight_360/wzk_vlog_beat_enhance1_track1238.mp4
VisBeatAssets/VideoSources/wzk_vlog_beat_enhance1_track1238/Versions/Source/Full/wzk_vlog_beat_enhance1_track1238.mp4
VisBeatAssets/VideoSources/zly2_0.mp4/.DS_Store
VisBeatAssets/VideoSources/zly2_0.mp4/VideoSource.json
VisBeatAssets/VideoSources/zly2_0.mp4/Data/.DS_Store
VisBeatAssets/VideoSources/zly2_0.mp4/Data/Backups/VideoSource.json
VisBeatAssets/VideoSources/zly2_0.mp4/Data/Features/video/directogram_powers/zly2_0_maxheight_360.pkl
VisBeatAssets/VideoSources/zly2_0.mp4/Versions/.DS_Store
VisBeatAssets/VideoSources/zly2_0.mp4/Versions/Original/maxheight_360/zly2_0.mp4
VisBeatAssets/VideoSources/zly2_0.mp4/Versions/Source/Full/zly2_0.mp4
VisBeatAssets/VideoSources/zly3_0.mp4/VideoSource.json
VisBeatAssets/VideoSources/zly3_0.mp4/Versions/Source/Full/zly3_0.mp4
bin/dancefer
visbeat3/.DS_Store
visbeat3/ADefines.py
visbeat3/AFileManager.py
visbeat3/AFuncDict.py
visbeat3/AImports.py
visbeat3/AObject.py
visbeat3/AParamDict.py
visbeat3/Audio.py
visbeat3/AudioClip.py
visbeat3/Event.py
visbeat3/EventList.py
visbeat3/Image.py
visbeat3/Image_CV.py
visbeat3/SourceLocationParser.py
visbeat3/TimeSignal.py
visbeat3/TimeSignal1D.py
visbeat3/VBMIDI.py
visbeat3/VBObject.py
visbeat3/Video.py
visbeat3/VideoClip.py
visbeat3/VideoSource.py
visbeat3/Video_CV.py
visbeat3/VisBeatDefines.py
visbeat3/VisBeatExampleVideo.py
visbeat3/VisBeatImports.py
visbeat3/VisualBeat.py
visbeat3/Warp.py
visbeat3/__init__.py
visbeat3/_dancefer_examples.py
visbeat3/_dancify_examples.py
visbeat3/_mediafiles.py
visbeat3/_music_examples.py
visbeat3/command_line.py
visbeat3.egg-info/PKG-INFO
visbeat3.egg-info/SOURCES.txt
visbeat3.egg-info/dependency_links.txt
visbeat3.egg-info/requires.txt
visbeat3.egg-info/top_level.txt
visbeat3/__pycache__/ADefines.cpython-37.pyc
visbeat3/__pycache__/AFileManager.cpython-37.pyc
visbeat3/__pycache__/AFuncDict.cpython-37.pyc
visbeat3/__pycache__/AImports.cpython-37.pyc
visbeat3/__pycache__/AObject.cpython-37.pyc
visbeat3/__pycache__/AParamDict.cpython-37.pyc
visbeat3/__pycache__/Audio.cpython-37.pyc
visbeat3/__pycache__/AudioClip.cpython-37.pyc
visbeat3/__pycache__/Event.cpython-37.pyc
visbeat3/__pycache__/EventList.cpython-37.pyc
visbeat3/__pycache__/Image.cpython-37.pyc
visbeat3/__pycache__/Image_CV.cpython-37.pyc
visbeat3/__pycache__/SourceLocationParser.cpython-37.pyc
visbeat3/__pycache__/TimeSignal.cpython-37.pyc
visbeat3/__pycache__/TimeSignal1D.cpython-37.pyc
visbeat3/__pycache__/VBObject.cpython-37.pyc
visbeat3/__pycache__/Video.cpython-37.pyc
visbeat3/__pycache__/VideoClip.cpython-37.pyc
visbeat3/__pycache__/VideoSource.cpython-37.pyc
visbeat3/__pycache__/Video_CV.cpython-37.pyc
visbeat3/__pycache__/VisBeatDefines.cpython-37.pyc
visbeat3/__pycache__/VisBeatExampleVideo.cpython-37.pyc
visbeat3/__pycache__/VisBeatImports.cpython-37.pyc
visbeat3/__pycache__/VisualBeat.cpython-37.pyc
visbeat3/__pycache__/Warp.cpython-37.pyc
visbeat3/__pycache__/__init__.cpython-37.pyc
visbeat3/__pycache__/_mediafiles.cpython-37.pyc
visbeat3/assets/audio/hit.wav
visbeat3/assets/images/VisBeatWatermark.png
visbeat3/fileui/__init__.py
visbeat3/fileui/uipath.py
visbeat3/fileui/__pycache__/__init__.cpython-37.pyc
visbeat3/fileui/__pycache__/uipath.cpython-37.pyc
visbeat3/vbgui/BeatGUI.py
visbeat3/vbgui/__init__.py
visbeat3/vbgui/__pycache__/BeatGUI.cpython-37.pyc
visbeat3/vbgui/__pycache__/__init__.cpython-37.pyc
================================================
FILE: src/video2npz/visbeat3/visbeat3.egg-info/dependency_links.txt
================================================
================================================
FILE: src/video2npz/visbeat3/visbeat3.egg-info/requires.txt
================================================
numpy
scipy
bs4
librosa
imageio
requests
moviepy
termcolor
youtube-dl
matplotlib
================================================
FILE: src/video2npz/visbeat3/visbeat3.egg-info/top_level.txt
================================================
visbeat3
================================================
FILE: videos/.gitkeep
================================================