Full Code of younggyoseo/FastTD3 for AI

main 7acc3a3c739d cached
25 files
738.3 KB
254.1k tokens
162 symbols
1 requests
Download .txt
Showing preview only (761K chars total). Download the full file or copy to clipboard to get everything.
Repository: younggyoseo/FastTD3
Branch: main
Commit: 7acc3a3c739d
Files: 25
Total size: 738.3 KB

Directory structure:
gitextract_7_u6euyd/

├── .gitignore
├── .pre-commit-config.yaml
├── LICENSE
├── README.md
├── data/
│   ├── humanoidbench_result.json
│   ├── isaaclab_result.json
│   └── playground_result.json
├── fast_td3/
│   ├── __init__.py
│   ├── environments/
│   │   ├── humanoid_bench_env.py
│   │   ├── isaaclab_env.py
│   │   ├── mtbench_env.py
│   │   └── mujoco_playground_env.py
│   ├── fast_td3.py
│   ├── fast_td3_deploy.py
│   ├── fast_td3_simbav2.py
│   ├── fast_td3_utils.py
│   ├── hyperparams.py
│   ├── train.py
│   ├── train_multigpu.py
│   └── training_notebook.ipynb
├── requirements/
│   ├── requirements.txt
│   ├── requirements_isaacgym.txt
│   └── requirements_playground.txt
├── setup.py
└── sim2real.md

================================================
FILE CONTENTS
================================================

================================================
FILE: .gitignore
================================================
models
wandb
*.pyc
.ipynb_checkpoints
fast_td3.egg-info/


================================================
FILE: .pre-commit-config.yaml
================================================
repos:
-   repo: https://github.com/psf/black
    rev: stable
    hooks:
    -   id: black 

================================================
FILE: LICENSE
================================================
This software is part of the BAIR Commons HIC Repository as of calendar year 2025.

--------------------------------------------------------------------------------

MIT License

Copyright (c) 2025 Younggyo Seo

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

--------------------------------------------------------------------------------

MIT License

Copyright (c) 2024 LeanRL developers

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

--------------------------------------------------------------------------------
Code in `cleanrl/ddpg_continuous_action.py` and `cleanrl/td3_continuous_action.py` are adapted from https://github.com/sfujim/TD3

MIT License

Copyright (c) 2020 Scott Fujimoto

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

--------------------------------------------------------------------------------
Code in `cleanrl/sac_continuous_action.py` is inspired and adapted from [haarnoja/sac](https://github.com/haarnoja/sac), [openai/spinningup](https://github.com/openai/spinningup), [pranz24/pytorch-soft-actor-critic](https://github.com/pranz24/pytorch-soft-actor-critic), [DLR-RM/stable-baselines3](https://github.com/DLR-RM/stable-baselines3), and [denisyarats/pytorch_sac](https://github.com/denisyarats/pytorch_sac).

- [haarnoja/sac](https://github.com/haarnoja/sac/blob/8258e33633c7e37833cc39315891e77adfbe14b2/LICENSE.txt)

COPYRIGHT

All contributions by the University of California:
Copyright (c) 2017, 2018 The Regents of the University of California (Regents)
All rights reserved.

All other contributions:
Copyright (c) 2017, 2018, the respective contributors
All rights reserved.

SAC uses a shared copyright model: each contributor holds copyright over
their contributions to the SAC codebase. The project versioning records all such
contribution and copyright details. If a contributor wants to further mark
their specific copyright on a particular contribution, they should indicate
their copyright solely in the commit message of the change when it is
committed.

LICENSE

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met: 

1. Redistributions of source code must retain the above copyright notice, this
   list of conditions and the following disclaimer. 
2. Redistributions in binary form must reproduce the above copyright notice,
   this list of conditions and the following disclaimer in the documentation
   and/or other materials provided with the distribution. 

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

CONTRIBUTION AGREEMENT

By contributing to the SAC repository through pull-request, comment,
or otherwise, the contributor releases their content to the
license and copyright terms herein.

- [openai/spinningup](https://github.com/openai/spinningup/blob/038665d62d569055401d91856abb287263096178/LICENSE)

The MIT License

Copyright (c) 2018 OpenAI (http://openai.com)

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

- [DLR-RM/stable-baselines3](https://github.com/DLR-RM/stable-baselines3/blob/44e53ff8115e8f4bff1d5218f10c8c7d1a4cfc12/LICENSE)

The MIT License

Copyright (c) 2019 Antonin Raffin

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

- [denisyarats/pytorch_sac](https://github.com/denisyarats/pytorch_sac/blob/81c5b536d3a1c5616b2531e446450df412a064fb/LICENSE)

MIT License

Copyright (c) 2019 Denis Yarats

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

- [pranz24/pytorch-soft-actor-critic](https://github.com/pranz24/pytorch-soft-actor-critic/blob/master/LICENSE)

MIT License

Copyright (c) 2018 Pranjal Tandon

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


---------------------------------------------------------------------------------
The CONTRIBUTING.md is adopted from https://github.com/entity-neural-network/incubator/blob/2a0c38b30828df78c47b0318c76a4905020618dd/CONTRIBUTING.md
and https://github.com/Stable-Baselines-Team/stable-baselines3-contrib/blob/master/CONTRIBUTING.md

MIT License

Copyright (c) 2021 Entity Neural Network developers

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.



MIT License

Copyright (c) 2020 Stable-Baselines Team

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


---------------------------------------------------------------------------------
The cleanrl/ppo_continuous_action_isaacgym.py is contributed by Nvidia

SPDX-FileCopyrightText: Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: MIT

Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.

--------------------------------------------------------------------------------

Code in `cleanrl/qdagger_dqn_atari_impalacnn.py` and `cleanrl/qdagger_dqn_atari_jax_impalacnn.py` are adapted from https://github.com/google-research/reincarnating_rl

**NOTE: the original repo did not fill out the copyright section in their license
so the following copyright notice is copied as is per the license requirement.
See https://github.com/google-research/reincarnating_rl/blob/a1d402f48a9f8658ca6aa0ddf416ab391745ff2c/LICENSE#L189


Copyright [yyyy] [name of copyright owner]

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.


================================================
FILE: README.md
================================================
# FastTD3 - Simple and Fast RL for Humanoid Control

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![arXiv](https://img.shields.io/badge/arXiv-2505.22642-b31b1b.svg)](https://arxiv.org/abs/2505.22642)


FastTD3 is a high-performance variant of the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, optimized for complex humanoid control tasks. FastTD3 can solve various humanoid control tasks with dexterous hands from HumanoidBench in just a few hours of training. Furthermore, FastTD3 achieves similar or better wall-time-efficiency to PPO in high-dimensional control tasks from popular simulations such as IsaacLab and MuJoCo Playground.

For more information, please see our [project webpage](https://younggyo.me/fast_td3)


## ❗ Updates

- **[Sep/20/2025]** Added `data` directory that contains training logs used for plotting in the report.

- **[Sep/17/2025]** Fixed an issue where `std_min` and `std_max` were not included in Actor config (credit: [@ningyuanz](https://github.com/ningyuanz)).

- **[Aug/07/2025]** If you encounter an issue in reproducing the performance, try disabling `torch.compile`. Please use `--no_compile` in argument.

- **[Jul/07/2025]** Added support for multi-GPU training! See [Multi-GPU Training](#multi-gpu-training) section for details. 

- **[Jul/02/2025]** Optimized codebase to speed up training around 10-30% when using a single RTX 4090 GPU.

- **[Jun/20/2025]** Added support for [MTBench](https://github.com/Viraj-Joshi/MTBench) with the help of [Viraj Joshi](https://viraj-joshi.github.io/).

- **[Jun/15/2025]** Added support for FastTD3 + [SimbaV2](https://dojeon-ai.github.io/SimbaV2/)! It's faster to train, and often achieves better asymptotic performance. We recommend using FastTD3 + SimbaV2 for most cases.

- **[Jun/06/2025]** Thanks to [Antonin Raffin](https://araffin.github.io/) ([@araffin](https://github.com/araffin)), we fixed the issues when using `n_steps` > 1, which stabilizes training with n-step return quite a lot!

- **[Jun/01/2025]** Updated the figures in the technical report to report deterministic evaluation for IsaacLab tasks.


## ✨ Features

FastTD3 offers researchers a significant speedup in training complex humanoid agents.

- Ready-to-go codebase with detailed instructions and pre-configured hyperparameters for each task
- Support popular benchmarks: HumanoidBench, MuJoCo Playground, and IsaacLab
- User-friendly features that can accelerate your research, such as rendering rollouts, torch optimizations (AMP and compile), and saving and loading checkpoints

## ⚙️ Prerequisites

Before you begin, ensure you have the following installed:
- Conda (for environment management)
- Git LFS (Large File Storage) -- For IsaacLab
- CMake -- For IsaacLab

And the following system packages:
```bash
sudo apt install libglfw3 libgl1-mesa-glx libosmesa6 git-lfs cmake
```

## 📖 Installation

This project requires different Conda environments for different sets of experiments.

### Common Setup
First, ensure the common dependencies are installed as mentioned in the [Prerequisites](#prerequisites) section.

### Environment for HumanoidBench

```bash
conda create -n fasttd3_hb -y python=3.10
conda activate fasttd3_hb
pip install --editable git+https://github.com/carlosferrazza/humanoid-bench.git#egg=humanoid-bench
pip install -r requirements/requirements.txt
```

### Environment for MuJoCo Playground
```bash
conda create -n fasttd3_playground -y python=3.10
conda activate fasttd3_playground
pip install -r requirements/requirements_playground.txt
```

**⚠️ Note:** Our `requirements_playground.txt` specifies `Jax==0.4.35`, which we found to be stable for latest GPUs in certain tasks such as `LeapCubeReorient` or `LeapCubeRotateZAxis`

**⚠️ Note:** Current FastTD3 codebase uses customized MuJoCo Playground that supports saving last observations into info dictionary. We will work on incorporating this change into official repository hopefully soon.

### Environment for IsaacLab
```bash
conda create -n fasttd3_isaaclab -y python=3.10
conda activate fasttd3_isaaclab

# Install IsaacLab (refer to official documentation for the latest steps)
# Official Quickstart: https://isaac-sim.github.io/IsaacLab/main/source/setup/quickstart.html
pip install 'isaacsim[all,extscache]==4.5.0' --extra-index-url https://pypi.nvidia.com
git clone https://github.com/isaac-sim/IsaacLab.git
cd IsaacLab
./isaaclab.sh --install
cd ..

# Install project-specific requirements
pip install -r requirements/requirements.txt
```

### Environment for MTBench
MTBench does not support humanoid experiments, but is a useful multi-task benchmark with massive parallel simulation. This could be useful for users who want to use FastTD3 for their multi-task experiments.

```bash
conda create -n fasttd3_mtbench -y python=3.8  # Note python version
conda activate fasttd3_mtbench

# Install IsaacGym -- recommend to follow instructions in https://github.com/BoosterRobotics/booster_gym
...

# Install MTBench
git clone https://github.com/Viraj-Joshi/MTBench.git
cd MTbench
pip install -e .
pip install skrl
cd ..

# Install project-specific requirements
pip install -r requirements/requirements_isaacgym.txt
```

### (Optional) Accelerate headless GPU rendering in cloud instances

In some cloud VM images the NVIDIA kernel driver is present but the user-space OpenGL/EGL/Vulkan libraries aren't, so MuJoCo falls back to CPU renderer. You can install just the NVIDIA user-space libraries (and skip rebuilding the kernel module) with:

```bash
sudo apt install -y kmod
sudo sh NVIDIA-Linux-x86_64-<your_driver_version>.run -s --no-kernel-module --ui=none --no-questions
```

As a rule-of-thumb, if you're running experiments and rendering is taking longer than 5 seconds, it is very likely that GPU renderer is not used.

## 🚀 Running Experiments

Activate the appropriate Conda environment before running experiments.

Please see `fast_td3/hyperparams.py` for information regarding hyperparameters!

### HumanoidBench Experiments
```bash
conda activate fasttd3_hb
# FastTD3
python fast_td3/train.py \
    --env_name h1hand-hurdle-v0 \
    --exp_name FastTD3 \
    --render_interval 5000 \
    --seed 1
# FastTD3 + SimbaV2
python fast_td3/train.py \
    --env_name h1hand-hurdle-v0 \
    --exp_name FastTD3 \
    --render_interval 5000 \
    --agent fasttd3_simbav2 \
    --batch_size 8192 \
    --critic_learning_rate_end 3e-5 \
    --actor_learning_rate_end 3e-5 \
    --weight_decay 0.0 \
    --critic_hidden_dim 512 \
    --critic_num_blocks 2 \
    --actor_hidden_dim 256 \
    --actor_num_blocks 1 \
    --seed 1
```

### MuJoCo Playground Experiments
```bash
conda activate fasttd3_playground
# FastTD3
python fast_td3/train.py \
    --env_name T1JoystickFlatTerrain \
    --exp_name FastTD3 \
    --render_interval 5000 \
    --seed 1
# FastTD3 + SimbaV2
python fast_td3/train.py \
    --env_name T1JoystickFlatTerrain \
    --exp_name FastTD3 \
    --render_interval 5000 \
    --agent fasttd3_simbav2 \
    --batch_size 8192 \
    --critic_learning_rate_end 3e-5 \
    --actor_learning_rate_end 3e-5 \
    --weight_decay 0.0 \
    --critic_hidden_dim 512 \
    --critic_num_blocks 2 \
    --actor_hidden_dim 256 \
    --actor_num_blocks 1 \
    --seed 1
```

### IsaacLab Experiments
```bash
conda activate fasttd3_isaaclab
# FastTD3
python fast_td3/train.py \
    --env_name Isaac-Velocity-Flat-G1-v0 \
    --exp_name FastTD3 \
    --render_interval 0 \
    --seed 1
# FastTD3 + SimbaV2
python fast_td3/train.py \
    --env_name Isaac-Repose-Cube-Allegro-Direct-v0 \
    --exp_name FastTD3 \
    --render_interval 0 \
    --agent fasttd3_simbav2 \
    --batch_size 8192 \
    --critic_learning_rate_end 3e-5 \
    --actor_learning_rate_end 3e-5 \
    --weight_decay 0.0 \
    --critic_hidden_dim 512 \
    --critic_num_blocks 2 \
    --actor_hidden_dim 256 \
    --actor_num_blocks 1 \
    --seed 1
```

### MTBench Experiments
```bash
conda activate fasttd3_mtbench
# FastTD3
python fast_td3/train.py \
    --env_name MTBench-meta-world-v2-mt10 \
    --exp_name FastTD3 \
    --render_interval 0 \
    --seed 1
# FastTD3 + SimbaV2
python fast_td3/train.py \
    --env_name MTBench-meta-world-v2-mt10 \
    --exp_name FastTD3 \
    --render_interval 0 \
    --agent fasttd3_simbav2 \
    --batch_size 8192 \
    --critic_learning_rate_end 3e-5 \
    --actor_learning_rate_end 3e-5 \
    --weight_decay 0.0 \
    --critic_hidden_dim 1024 \
    --critic_num_blocks 2 \
    --actor_hidden_dim 512 \
    --actor_num_blocks 1 \
    --seed 1
```

**Quick note:** For boolean-based arguments, you can set them to False by adding `no_` in front each argument, for instance, if you want to disable Clipped Q Learning, you can specify `--no_use_cdq` in your command.

## 💡 Performance-Related Tips

We used a single Nvidia A100 80GB GPU for all experiments. Here are some remarks and tips for improving performances in your setup or troubleshooting in your machine configurations.

- *Sample-efficiency* tends to improve with larger `num_envs`, `num_updates`, and `batch_size`. But this comes at the cost of *Time-efficiency*. Our default settings are optimized for wall-time efficiency on a single A100 80GB GPU. If you're using a different setup, consider tuning hyperparameters accordingly.
- When FastTD3 performance is stuck at local minima at the early phase of training in your experiments
  - First consider increasing the `num_updates`. This happens usually when the agent fails to exploit value functions. We also find higher `num_updates` tends to be helpful for relatively easier tasks or tasks with low-dimensional action spaces.
  - If the agent is completely stuck or much worse than your expectation, try using `num_steps=3` or disabling `use_cdq`.
  - For tasks that have penalty reward terms (e.g., torques, energy, action_rate, ..), consider lowering them for initial experiments, and tune the values. In some cases, curriculum learning with lower penalty terms followed by fine-tuning with stronger terms is effective.
- When you encounter out-of-memory error with your GPU, our recommendation for reducing GPU usage is (i) smaller `buffer_size`, (ii) smaller `batch_size`, and then (iii) smaller `num_envs`. Because our codebase is assigning the whole replay buffer in GPU to reduce CPU-GPU transfer bottleneck, it usually has the largest GPU consumption, but usually less harmful to reduce.
- Consider using `--compile_mode max-autotune` if you plan to run for many training steps. This may speed up training by up to 10% at the cost of a few additional minutes of heavy compilation.

## Multi-GPU Training
We support multi-GPU training. If your machine supports multiple GPUs, or specify multiple GPUs using `CUDA_VISIBLE_DEVICES`, and run `train_multigpu.py`, it will automatically use all GPUs to scale up training.

**Important:** Our multi-GPU implementation launches the **same experiment independently on each GPU** rather than distributing parameters across GPUs. This means:
- Effective number of environments: `num_envs × num_gpus`
- Effective batch size: `batch_size × num_gpus` 
- Effective buffer size: `buffer_size × num_gpus`

Each GPU runs a complete copy of the training process, which scales up data collection and training throughput proportionally to the number of GPUs.

For instance, running IsaacLab experiments with 4 GPUs and `num_envs=1024` will end up in similar results as experiments with 1 GPU with `num_envs=4096`.

## 🛝 Playing with the FastTD3 training

A Jupyter notebook (`training_notebook.ipynb`) is available to help you get started with:
- Training FastTD3 agents.
- Loading pre-trained models.
- Visualizing agent behavior.
- Potentially, re-training or fine-tuning models.

## 🤖 Sim-to-Real RL with FastTD3

We provide the [walkthrough](sim2real.md) for training deployable policies with FastTD3.

## Contributing

We welcome contributions! Please feel free to submit issues and pull requests.

## License

This project is licensed under the MIT License -- see the [LICENSE](LICENSE) file for details. Note that the repository relies on third-party libraries subject to their respective licenses.

## Acknowledgements

This codebase builds upon [LeanRL](https://github.com/pytorch-labs/LeanRL) framework. 

We would like to thank people who have helped throughout the project:

- We thank [Kevin Zakka](https://kzakka.com/) for the help in setting up MuJoCo Playground.
- We thank [Changyeon Kim](https://changyeon.site/) for testing the early version of this codebase

## Citations

### FastTD3
```bibtex
@article{seo2025fasttd3,
  title={FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Control},
  author={Seo, Younggyo and Sferrazza, Carmelo and Geng, Haoran and Nauman, Michal and Yin, Zhao-Heng and Abbeel, Pieter},
  journal={arXiv preprint arXiv:2505.22642},
  year={2025}
}
```

### TD3
```bibtex
@inproceedings{fujimoto2018addressing,
  title={Addressing function approximation error in actor-critic methods},
  author={Fujimoto, Scott and Hoof, Herke and Meger, David},
  booktitle={International conference on machine learning},
  pages={1587--1596},
  year={2018},
  organization={PMLR}
}
```

### SimbaV2
```bibtex
@article{lee2025hyperspherical,
  title={Hyperspherical normalization for scalable deep reinforcement learning},
  author={Lee, Hojoon and Lee, Youngdo and Seno, Takuma and Kim, Donghu and Stone, Peter and Choo, Jaegul},
  journal={arXiv preprint arXiv:2502.15280},
  year={2025}
}
```

### LeanRL

Following the [LeanRL](https://github.com/pytorch-labs/LeanRL)'s recommendation, we put CleanRL's bibtex here:

```bibtex
@article{huang2022cleanrl,
  author  = {Shengyi Huang and Rousslan Fernand Julien Dossa and Chang Ye and Jeff Braga and Dipam Chakraborty and Kinal Mehta and João G.M. Araújo},
  title   = {CleanRL: High-quality Single-file Implementations of Deep Reinforcement Learning Algorithms},
  journal = {Journal of Machine Learning Research},
  year    = {2022},
  volume  = {23},
  number  = {274},
  pages   = {1--18},
  url     = {http://jmlr.org/papers/v23/21-1342.html}
}
```

### Parallel Q-Learning (PQL)
```bibtex
@inproceedings{li2023parallel,
  title={Parallel $ Q $-Learning: Scaling Off-policy Reinforcement Learning under Massively Parallel Simulation},
  author={Li, Zechu and Chen, Tao and Hong, Zhang-Wei and Ajay, Anurag and Agrawal, Pulkit},
  booktitle={International Conference on Machine Learning},
  pages={19440--19459},
  year={2023},
  organization={PMLR}
}
```

### HumanoidBench
```bibtex
@inproceedings{sferrazza2024humanoidbench,
  title={Humanoidbench: Simulated humanoid benchmark for whole-body locomotion and manipulation},
  author={Sferrazza, Carmelo and Huang, Dun-Ming and Lin, Xingyu and Lee, Youngwoon and Abbeel, Pieter},
  booktitle={Robotics: Science and Systems},
  year={2024}
}
```

### MuJoCo Playground
```bibtex
@article{zakka2025mujoco,
  title={MuJoCo Playground},
  author={Zakka, Kevin and Tabanpour, Baruch and Liao, Qiayuan and Haiderbhai, Mustafa and Holt, Samuel and Luo, Jing Yuan and Allshire, Arthur and Frey, Erik and Sreenath, Koushil and Kahrs, Lueder A and others},
  journal={arXiv preprint arXiv:2502.08844},
  year={2025}
}
```

### IsaacLab
```bibtex
@article{mittal2023orbit,
   author={Mittal, Mayank and Yu, Calvin and Yu, Qinxi and Liu, Jingzhou and Rudin, Nikita and Hoeller, David and Yuan, Jia Lin and Singh, Ritvik and Guo, Yunrong and Mazhar, Hammad and Mandlekar, Ajay and Babich, Buck and State, Gavriel and Hutter, Marco and Garg, Animesh},
   journal={IEEE Robotics and Automation Letters},
   title={Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments},
   year={2023},
   volume={8},
   number={6},
   pages={3740-3747},
   doi={10.1109/LRA.2023.3270034}
}
```

### MTBench
```bibtex
@inproceedings{
joshi2025benchmarking,
title={Benchmarking Massively Parallelized Multi-Task Reinforcement Learning for Robotics Tasks},
author={Viraj Joshi and Zifan Xu and Bo Liu and Peter Stone and Amy Zhang},
booktitle={Reinforcement Learning Conference},
year={2025},
url={https://openreview.net/forum?id=z0MM0y20I2}
}
```

### Getting SAC to Work on a Massive Parallel Simulator
```bibtex
@article{raffin2025isaacsim,
  title   = "Getting SAC to Work on a Massive Parallel Simulator: An RL Journey With Off-Policy Algorithms",
  author  = "Raffin, Antonin",
  journal = "araffin.github.io",
  year    = "2025",
  month   = "Feb",
  url     = "https://araffin.github.io/post/sac-massive-sim/"
}
```

### Speeding Up SAC with Massively Parallel Simulation
```bibtex
@article{shukla2025fastsac,
  title   = "Speeding Up SAC with Massively Parallel Simulation",
  author  = "Shukla, Arth",
  journal = "https://arthshukla.substack.com",
  year    = "2025",
  month   = "Mar",
  url     = "https://arthshukla.substack.com/p/speeding-up-sac-with-massively-parallel"
}
```


================================================
FILE: data/humanoidbench_result.json
================================================
{
  "h1hand_walk": {
    "time": [
      0.0,
      327.4388405844,
      654.8776811688,
      982.3165217531999,
      1309.7553623376,
      1637.194202922,
      1964.6330435063999,
      2292.0718840908,
      2619.5107246752,
      2946.9495652596,
      3274.388405844
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0
    ],
    "return": [
      0.0,
      18.961605707804363,
      21.807273546854656,
      48.724927266438804,
      94.65358352661133,
      178.67118326822916,
      501.64170328776044,
      711.7367757161459,
      901.3774210611979,
      928.8246256510416,
      928.2600301106771
    ],
    "return_min": [
      0.0,
      14.858372081304717,
      12.96401166123347,
      41.67681180605566,
      54.95595528974483,
      94.37760201041398,
      279.43906161791847,
      545.4619319542776,
      877.1935439686172,
      921.6425265540179,
      906.3836415708065
    ],
    "return_max": [
      0.0,
      23.06483933430401,
      30.65053543247584,
      55.77304272682195,
      134.35121176347783,
      262.96476452604435,
      723.8443449576024,
      878.0116194780142,
      925.5612981537786,
      936.0067247480654,
      950.1364186505477
    ]
  },
  "h1hand_stand": {
    "time": [
      0.0,
      325.5565317698,
      651.1130635396,
      976.6695953094,
      1302.2261270792,
      1627.782658849,
      1953.3391906188,
      2278.8957223886,
      2604.4522541584,
      2930.0087859282003,
      3255.565317698
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0
    ],
    "return": [
      0.0,
      31.879506429036457,
      42.487387339274086,
      53.94635899861654,
      71.63724517822266,
      109.86285400390625,
      255.9549357096354,
      628.9070638020834,
      776.9771118164062,
      887.54248046875,
      911.9251912434896
    ],
    "return_min": [
      0.0,
      21.445549292274528,
      22.163247088006734,
      45.241289229102975,
      64.28271961875848,
      76.26589758620838,
      191.1954024340911,
      393.33148063655887,
      656.7962055436002,
      851.7788577992328,
      880.8326840828662
    ],
    "return_max": [
      0.0,
      42.313463565798386,
      62.81152759054144,
      62.6514287681301,
      78.99177073768683,
      143.45981042160412,
      320.7144689851797,
      864.4826469676079,
      897.1580180892123,
      923.3061031382672,
      943.017698404113
    ]
  },
  "h1hand_run": {
    "time": [
      0.0,
      306.4421410813333,
      612.8842821626666,
      919.326423244,
      1225.7685643253333,
      1532.2107054066666,
      1838.652846488,
      2145.0949875693336,
      2451.5371286506665,
      2757.9792697320004,
      3064.4214108133333,
      3370.8635518946667,
      3677.305692976,
      3983.747834057334,
      4290.189975138667,
      4596.63211622
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0
    ],
    "return": [
      0.0,
      6.890847841898601,
      10.521868069966635,
      9.320551554361979,
      17.587395985921223,
      31.98662503560384,
      46.11953608194987,
      74.0572992960612,
      121.98380025227864,
      210.85086059570312,
      506.30312093098956,
      524.246826171875,
      775.0013631184896,
      811.9727172851562,
      886.3514200846354,
      902.0192260742188
    ],
    "return_min": [
      0.0,
      5.2453829775246215,
      9.646580055146266,
      6.430592906343505,
      12.881361282168676,
      26.82687314527471,
      41.16444668808793,
      62.03005710131488,
      100.789247637638,
      183.66015405980465,
      427.02318928000045,
      408.7644706774459,
      737.4298733834557,
      774.6940229887994,
      879.065567239731,
      892.5835266487527
    ],
    "return_max": [
      0.0,
      8.536312706272579,
      11.397156084787003,
      12.210510202380451,
      22.29343068967377,
      37.14637692593297,
      51.07462547581181,
      86.08454149080752,
      143.17835286691928,
      238.0415671316016,
      585.5830525819787,
      639.729181666304,
      812.5728528535235,
      849.2514115815131,
      893.6372729295398,
      911.4549254996848
    ]
  },
  "h1hand_reach": {
    "time": [
      0.0,
      372.80056838400003,
      745.6011367680001,
      1118.401705152,
      1491.2022735360001,
      1864.00284192,
      2236.803410304,
      2609.603978688,
      2982.4045470720002,
      3355.2051154560004,
      3728.00568384,
      4100.806252224001,
      4473.606820608,
      4846.407388992,
      5219.207957376,
      5592.00852576,
      5964.8090941440005,
      6337.609662528001,
      6710.410230912001,
      7083.210799296,
      7456.01136768,
      7828.811936064,
      8201.612504448001,
      8574.413072832,
      8947.213641216,
      9320.0142096,
      9692.814777984,
      10065.615346368,
      10438.415914752,
      10811.216483136,
      11184.01705152
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0,
      13440000.0,
      14080000.0,
      14720000.0,
      15360000.0,
      16000000.0,
      16640000.0,
      17280000.0,
      17920000.0,
      18560000.0,
      19200000.0
    ],
    "return": [
      0.0,
      941.600341796875,
      1135.4913330078125,
      1474.2825520833333,
      2313.754109700521,
      3213.2079264322915,
      4587.813883463542,
      5064.874186197917,
      5511.294270833333,
      5576.633138020833,
      6172.243489583333,
      7008.345377604167,
      6894.714680989583,
      7536.7392578125,
      7494.722819010417,
      7460.627278645833,
      7870.114095052083,
      8164.121256510417,
      8070.581217447917,
      8084.484700520833,
      8364.873697916666,
      8254.95654296875,
      8171.631673177083,
      8403.752604166666,
      8585.269368489584,
      8388.333170572916,
      8449.694010416666,
      8608.694986979166,
      8495.815755208334,
      8748.194661458334,
      8146.561197916667
    ],
    "return_min": [
      0.0,
      681.8662271797355,
      1021.0384348384862,
      1397.7603324284514,
      1976.0678889906676,
      2504.978510893123,
      3452.5915196733913,
      4091.507234596569,
      4614.513790451145,
      4584.797954406994,
      5541.806414474982,
      6259.515364983769,
      6343.318836978547,
      6854.539919974239,
      6980.611797856333,
      6705.693438934862,
      7218.966063142879,
      7807.798913658527,
      7519.692863858716,
      7372.049219729361,
      7880.456149452872,
      7570.937682802555,
      7367.755272289276,
      7989.806520979583,
      8098.439330032555,
      7945.363999112206,
      8019.104933439191,
      8156.065950744518,
      7987.549893436531,
      8366.520573819495,
      7681.10435210547
    ],
    "return_max": [
      0.0,
      1201.3344564140145,
      1249.9442311771388,
      1550.804771738215,
      2651.4403304103744,
      3921.4373419714602,
      5723.036247253693,
      6038.241137799265,
      6408.074751215521,
      6568.468321634672,
      6802.680564691684,
      7757.175390224565,
      7446.110525000619,
      8218.938595650761,
      8008.8338401645005,
      8215.561118356805,
      8521.262126961288,
      8520.443599362306,
      8621.469571037118,
      8796.920181312305,
      8849.29124638046,
      8938.975403134946,
      8975.50807406489,
      8817.69868735375,
      9072.099406946612,
      8831.302342033627,
      8880.283087394142,
      9061.324023213814,
      9004.081616980136,
      9129.868749097173,
      8612.018043727863
    ]
  },
  "h1hand_hurdle": {
    "time": [
      0.0,
      361.5054530611936,
      723.0109061223872,
      1084.5163591835806,
      1446.0218122447743,
      1807.5272653059678,
      2169.032718367161,
      2530.538171428355,
      2892.0436244895486,
      3253.549077550742,
      3615.0545306119357,
      3976.559983673129,
      4338.065436734322,
      4699.570889795516,
      5061.07634285671,
      5422.581795917904,
      5784.087248979097,
      6145.5927020402905,
      6507.098155101484,
      6868.603608162677,
      7230.109061223871,
      7591.614514285065,
      7953.119967346258,
      8314.625420407452,
      8676.130873468644,
      9037.636326529839,
      9399.141779591033,
      9760.647232652227,
      10122.15268571342,
      10483.658138774614,
      10845.163591835808,
      11206.669044897
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0,
      13440000.0,
      14080000.0,
      14720000.0,
      15360000.0,
      16000000.0,
      16640000.0,
      17280000.0,
      17920000.0,
      18560000.0,
      19200000.0,
      19840000.0
    ],
    "return": [
      0.0,
      6.492611567179362,
      10.588607947031656,
      18.610817273457844,
      29.661352157592773,
      49.00843048095703,
      85.58958435058594,
      115.55878194173177,
      173.2019500732422,
      218.64263916015625,
      319.6346893310547,
      377.22975667317706,
      496.73987833658856,
      490.29189046223956,
      610.6637573242188,
      658.8320719401041,
      689.0982869466146,
      710.2075602213541,
      735.3473714192709,
      778.7053426106771,
      762.7415568033854,
      804.9999593098959,
      830.8478800455729,
      852.31982421875,
      861.3894653320312,
      814.1538696289062,
      871.5404663085938,
      876.4856160481771,
      852.7265625,
      889.6229248046875,
      887.1525065104166,
      886.8323160807291
    ],
    "return_min": [
      0.0,
      6.225973005334275,
      8.412515126451241,
      15.738143769806989,
      21.76995365926382,
      31.328784883872007,
      56.83435619834776,
      79.92347525493489,
      112.07297356902194,
      138.09470906365925,
      216.08342306974944,
      259.30040536744895,
      389.9528713112492,
      378.8689856869445,
      480.1631532334461,
      565.0675789124052,
      598.9966247159923,
      618.670444534875,
      659.2942580856163,
      729.4083980816131,
      708.118056513654,
      737.1563442724927,
      785.5860491473233,
      822.1971602106087,
      853.2213237200906,
      798.9233122130538,
      848.562144860611,
      867.3290162790252,
      808.398990047507,
      877.0273627520126,
      880.6210154175322,
      878.8996722928737
    ],
    "return_max": [
      0.0,
      6.7592501290244495,
      12.764700767612071,
      21.4834907771087,
      37.55275065592173,
      66.68807607804206,
      114.34481250282411,
      151.19408862852865,
      234.33092657746244,
      299.19056925665325,
      423.18595559235996,
      495.1591079789052,
      603.526885361928,
      601.7147952375346,
      741.1643614149914,
      752.596564967803,
      779.1999491772369,
      801.7446759078333,
      811.4004847529254,
      828.0022871397412,
      817.3650570931168,
      872.8435743472991,
      876.1097109438225,
      882.4424882268913,
      869.5576069439719,
      829.3844270447587,
      894.5187877565764,
      885.6422158173291,
      897.054134952493,
      902.2184868573624,
      893.683997603301,
      894.7649598685846
    ]
  },
  "h1hand_crawl": {
    "time": [
      0.0,
      351.0455393933077,
      702.0910787866154,
      1053.136618179923,
      1404.1821575732308,
      1755.2276969665386,
      2106.273236359846,
      2457.318775753154,
      2808.3643151464616,
      3159.4098545397696,
      3510.455393933077,
      3861.5009333263847,
      4212.546472719692,
      4563.592012113
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0
    ],
    "return": [
      0.0,
      352.52854410807294,
      560.5812377929688,
      552.0515441894531,
      642.7318115234375,
      714.5783081054688,
      806.6729939778646,
      860.4519856770834,
      892.0007731119791,
      920.3323160807291,
      924.2599487304688,
      945.3890991210938,
      948.0378011067709,
      956.2367553710938
    ],
    "return_min": [
      0.0,
      324.2043638986621,
      545.5137620982706,
      483.4934811974669,
      635.1979678248894,
      683.6522397156278,
      727.5537564989331,
      784.8352581603768,
      839.2658061728478,
      887.326843488466,
      905.17119209457,
      927.7121326375325,
      932.2496491343232,
      945.4639700799313
    ],
    "return_max": [
      0.0,
      380.85272431748376,
      575.6487134876669,
      620.6096071814393,
      650.2656552219856,
      745.5043764953097,
      885.7922314567961,
      936.0687131937899,
      944.7357400511105,
      953.3377886729922,
      943.3487053663675,
      963.066065604655,
      963.8259530792185,
      967.0095406622562
    ]
  },
  "h1hand_maze": {
    "time": [
      0.0,
      373.7336517353571,
      747.4673034707142,
      1121.2009552060715,
      1494.9346069414285,
      1868.6682586767856,
      2242.401910412143,
      2616.1355621475,
      2989.869213882857,
      3363.6028656182143,
      3737.3365173535713,
      4111.070169088928,
      4484.803820824286,
      4858.537472559643,
      5232.271124295
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0
    ],
    "return": [
      0.0,
      140.10218302408853,
      145.8614705403646,
      147.9732869466146,
      182.3691864013672,
      233.79485066731772,
      329.2705078125,
      352.1261393229167,
      357.1931559244792,
      357.9311014811198,
      298.9239857991536,
      358.42406209309894,
      350.0440165201823,
      353.48012288411456,
      358.6817118326823
    ],
    "return_min": [
      0.0,
      124.88956930471686,
      135.93298616192584,
      142.26980814418167,
      163.01049749771548,
      169.52534951133262,
      312.57792152579833,
      341.7712339158121,
      350.91098197823186,
      354.19958286243155,
      211.5847442406885,
      346.7498400009268,
      342.2860426951425,
      344.2272969649359,
      354.4987809321401
    ],
    "return_max": [
      0.0,
      155.31479674346022,
      155.78995491880335,
      153.67676574904752,
      201.7278753050189,
      298.0643518233028,
      345.96309409920167,
      362.48104473002127,
      363.4753298707265,
      361.66262009980807,
      386.26322735761875,
      370.0982841852711,
      357.8019903452221,
      362.73294880329325,
      362.8646427332245
    ]
  },
  "h1hand_sit_simple": {
    "time": [
      0.0,
      352.13274472366663,
      704.2654894473333,
      1056.398234171,
      1408.5309788946665,
      1760.6637236183333,
      2112.796468342,
      2464.9292130656668,
      2817.061957789333,
      3169.194702513
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0
    ],
    "return": [
      0.0,
      139.5462900797526,
      653.7905069986979,
      863.320068359375,
      937.2736612955729,
      917.0751342773438,
      925.4646809895834,
      934.3729451497396,
      931.1214192708334,
      929.0094401041666
    ],
    "return_min": [
      0.0,
      39.1384600509537,
      561.3652721120882,
      840.7680414731517,
      928.7285812113073,
      882.5101152312143,
      910.9714846547159,
      910.6220791649206,
      912.940393490759,
      909.0497473763742
    ],
    "return_max": [
      0.0,
      239.95412010855148,
      746.2157418853076,
      885.8720952455983,
      945.8187413798385,
      951.6401533234732,
      939.9578773244508,
      958.1238111345587,
      949.3024450509077,
      948.9691328319591
    ]
  },
  "h1hand_sit_hard": {
    "time": [
      0.0,
      355.06120608526663,
      710.1224121705333,
      1065.1836182558,
      1420.2448243410665,
      1775.3060304263333,
      2130.3672365116,
      2485.4284425968663,
      2840.489648682133,
      3195.5508547674003,
      3550.6120608526667,
      3905.673266937933,
      4260.7344730232,
      4615.795679108467,
      4970.8568851937325,
      5325.918091279,
      5680.979297364266,
      6036.040503449533,
      6391.101709534801,
      6746.162915620066,
      7101.224121705333,
      7456.2853277906,
      7811.346533875866,
      8166.407739961132,
      8521.4689460464,
      8876.530152131665,
      9231.591358216934,
      9586.6525643022,
      9941.713770387465,
      10296.774976472732,
      10651.836182558
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0,
      13440000.0,
      14080000.0,
      14720000.0,
      15360000.0,
      16000000.0,
      16640000.0,
      17280000.0,
      17920000.0,
      18560000.0,
      19200000.0
    ],
    "return": [
      0.0,
      95.34047635396321,
      195.36861419677734,
      318.97853597005206,
      396.64198811848956,
      375.91650390625,
      488.83921305338544,
      548.6920267740885,
      574.7945251464844,
      566.6195780436198,
      583.4102783203125,
      590.7933756510416,
      600.908457438151,
      594.9339090983073,
      610.3661804199219,
      616.0959269205729,
      610.0286865234375,
      601.2849934895834,
      611.8743286132812,
      634.0820617675781,
      632.5235493977865,
      669.2791239420573,
      701.1968180338541,
      683.4253133138021,
      706.3424275716146,
      741.2285461425781,
      699.800537109375,
      749.2343546549479,
      752.3892517089844,
      734.4484252929688,
      726.0768636067709
    ],
    "return_min": [
      0.0,
      2.5724704693975298,
      93.58268813745721,
      233.71247327846993,
      368.7017851150339,
      332.2081460365896,
      340.63550191707344,
      329.6445384869926,
      351.03336679701863,
      343.03801256893826,
      363.99036303185073,
      360.8858716647696,
      369.6746990033322,
      375.6881012599298,
      384.3501951686797,
      391.8689483225843,
      385.04690037041826,
      394.37185439974064,
      381.77529344069296,
      411.24672410522163,
      405.85462333371214,
      459.9114779257408,
      504.43995094660033,
      483.4763839716003,
      499.2238858422869,
      538.4201845543151,
      470.58627615990247,
      533.4576585016802,
      541.3393913458676,
      524.7727664466893,
      470.0900042127373
    ],
    "return_max": [
      0.0,
      188.1084822385289,
      297.15454025609745,
      404.2445986616342,
      424.58219112194524,
      419.6248617759104,
      637.0429241896975,
      767.7395150611844,
      798.5556834959501,
      790.2011435183013,
      802.8301936087743,
      820.7008796373136,
      832.1422158729698,
      814.1797169366847,
      836.3821656711641,
      840.3229055185615,
      835.0104726764567,
      808.1981325794261,
      841.9733637858695,
      856.9173994299347,
      859.1924754618608,
      878.6467699583736,
      897.9536851211079,
      883.374242656004,
      913.4609693009423,
      944.0369077308411,
      929.0147980588475,
      965.0110508082156,
      963.4391120721011,
      944.1240841392482,
      982.0637230008044
    ]
  },
  "h1hand_balance_simple": {
    "time": [
      0.0,
      175.717594372775,
      351.43518874555,
      527.152783118325,
      702.8703774911,
      878.587971863875,
      1054.30556623665,
      1230.023160609425,
      1405.7407549822,
      1581.458349354975,
      1757.17594372775,
      1932.8935381005253,
      2108.6111324733,
      2284.328726846075,
      2460.04632121885,
      2635.763915591625,
      2811.4815099644,
      2987.1991043371754,
      3162.91669870995,
      3338.6342930827254,
      3514.3518874555,
      3690.0694818282755,
      3865.7870762010507,
      4041.5046705738255,
      4217.2222649466,
      4392.939859319375,
      4568.65745369215,
      4744.375048064925,
      4920.0926424377,
      5095.8102368104755,
      5271.52783118325,
      5447.245425556025,
      5622.9630199288,
      5798.680614301576,
      5974.398208674351,
      6150.115803047125,
      6325.8333974199,
      6501.550991792676,
      6677.268586165451,
      6852.986180538226,
      7028.703774911
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0,
      13440000.0,
      14080000.0,
      14720000.0,
      15360000.0,
      16000000.0,
      16640000.0,
      17280000.0,
      17920000.0,
      18560000.0,
      19200000.0,
      19840000.0,
      20480000.0,
      21120000.0,
      21760000.0,
      22400000.0,
      23040000.0,
      23680000.0,
      24320000.0,
      24960000.0,
      25600000.0
    ],
    "return": [
      0.0,
      52.67643197377523,
      77.08096504211426,
      100.20963668823242,
      118.43112309773763,
      157.09767150878906,
      208.31290181477866,
      215.8346757888794,
      295.65764872233075,
      373.72972615559894,
      395.9920959472656,
      326.1277338663737,
      487.8640950520833,
      412.4946568806966,
      481.59149678548175,
      489.4783935546875,
      555.3950398763021,
      549.860850016276,
      587.4115397135416,
      623.5388387044271,
      538.5026601155599,
      437.4591878255208,
      563.7863718668619,
      591.0351969401041,
      635.8733622233073,
      531.7674204508463,
      557.8404591878256,
      567.099183400472,
      504.17720794677734,
      611.8459065755209,
      568.1315002441406,
      531.0865529378256,
      719.8113403320312,
      617.6780344645182,
      661.9119974772135,
      701.4621276855469,
      612.2174580891927,
      698.0676879882812,
      607.3912162780762,
      701.1986897786459,
      788.4394226074219
    ],
    "return_min": [
      0.0,
      34.03153521489918,
      67.34714304892007,
      93.43179041853405,
      108.3451803854909,
      131.60774027010135,
      186.807812935322,
      121.50917666317413,
      243.0843846608089,
      314.6589689026109,
      335.93710664802484,
      199.6418475452607,
      421.46378419781684,
      271.96722431041565,
      419.1856732133755,
      384.7529814203755,
      496.47070280779275,
      468.9030197426357,
      520.0155579034114,
      555.5749497533352,
      406.5754864025145,
      275.34879196835936,
      498.0399353927207,
      550.1356761025373,
      516.3616959889615,
      320.4064699483651,
      442.36312091556783,
      326.04143673322324,
      303.6744638373668,
      485.79766687848047,
      303.55332555045385,
      466.7197913683005,
      625.171113211483,
      395.48405865968545,
      542.6871855684739,
      615.614612025825,
      499.65604515073494,
      548.4941266431731,
      350.1081528445121,
      573.6862052997205,
      653.9984031207549
    ],
    "return_max": [
      0.0,
      71.32132873265128,
      86.81478703530844,
      106.9874829579308,
      128.51706580998433,
      182.58760274747678,
      229.8179906942353,
      310.1601749145847,
      348.2309127838526,
      432.80048340858696,
      456.0470852465064,
      452.61362018748673,
      554.2644059063498,
      553.0220894509775,
      543.997320357588,
      594.2038056889995,
      614.3193769448116,
      630.8186802899163,
      654.8075215236719,
      691.502727655519,
      670.4298338286053,
      599.5695836826823,
      629.5328083410031,
      631.9347177776709,
      755.385028457653,
      743.1283709533275,
      673.3177974600833,
      808.1569300677209,
      704.6799520561879,
      737.8941462725612,
      832.7096749378275,
      595.4533145073506,
      814.4515674525795,
      839.872010269351,
      781.1368093859531,
      787.3096433452688,
      724.7788710276506,
      847.6412493333894,
      864.6742797116402,
      828.7111742575712,
      922.8804420940888
    ]
  },
  "h1hand_balance_hard": {
    "time": [
      0.0,
      181.3574838361809,
      362.7149676723618,
      544.0724515085427,
      725.4299353447236,
      906.7874191809045,
      1088.1449030170854,
      1269.5023868532662,
      1450.8598706894472,
      1632.2173545256283,
      1813.574838361809,
      1994.9323221979898,
      2176.289806034171,
      2357.647289870352,
      2539.0047737065324,
      2720.3622575427135,
      2901.7197413788945,
      3083.0772252150755,
      3264.4347090512565,
      3445.792192887437,
      3627.149676723618,
      3808.507160559799,
      3989.8646443959797,
      4171.222128232161,
      4352.579612068342,
      4533.937095904523,
      4715.294579740704,
      4896.652063576884,
      5078.009547413065,
      5259.367031249247,
      5440.724515085427,
      5622.081998921608,
      5803.439482757789,
      5984.79696659397,
      6166.154450430151,
      6347.511934266332,
      6528.869418102513,
      6710.226901938693,
      6891.584385774874,
      7072.941869611055,
      7254.299353447236,
      7435.656837283417,
      7617.014321119598,
      7798.371804955779,
      7979.729288791959,
      8161.08677262814,
      8342.444256464321,
      8523.801740300501,
      8705.159224136683,
      8886.516707972865,
      9067.874191809045,
      9249.231675645227,
      9430.589159481407,
      9611.946643317588,
      9793.304127153768,
      9974.66161098995,
      10156.01909482613,
      10337.376578662312,
      10518.734062498494,
      10700.091546334672,
      10881.449030170854,
      11062.806514007034,
      11244.163997843216,
      11425.521481679396,
      11606.878965515578,
      11788.236449351758,
      11969.59393318794,
      12150.951417024122,
      12332.308900860302,
      12513.666384696484,
      12695.023868532664,
      12876.381352368846,
      13057.738836205026,
      13239.096320041204,
      13420.453803877386,
      13601.811287713566,
      13783.168771549748,
      13964.526255385928,
      14145.88373922211,
      14327.24122305829,
      14508.598706894472,
      14689.956190730652,
      14871.313674566834,
      15052.671158403016,
      15234.028642239196,
      15415.386126075378,
      15596.743609911558,
      15778.10109374774,
      15959.458577583919,
      16140.816061420099,
      16322.17354525628,
      16503.53102909246,
      16684.888512928643,
      16866.245996764825,
      17047.603480601003,
      17228.960964437185,
      17410.318448273367,
      17591.67593210955,
      17773.03341594573,
      17954.39089978191,
      18135.74838361809,
      18317.105867454273,
      18498.463351290455,
      18679.820835126633,
      18861.178318962815,
      19042.535802798993,
      19223.893286635175,
      19405.250770471357,
      19586.608254307535,
      19767.965738143717,
      19949.3232219799,
      20130.68070581608,
      20312.03818965226,
      20493.39567348844,
      20674.753157324623,
      20856.110641160805,
      21037.468124996987,
      21218.825608833165,
      21400.183092669344,
      21581.540576505526,
      21762.898060341708,
      21944.25554417789,
      22125.613028014068,
      22306.97051185025,
      22488.32799568643,
      22669.685479522614,
      22851.042963358792,
      23032.400447194974,
      23213.757931031156,
      23395.115414867338,
      23576.472898703516,
      23757.830382539698,
      23939.18786637588,
      24120.54535021206,
      24301.902834048244,
      24483.260317884422,
      24664.617801720604,
      24845.975285556786,
      25027.332769392968,
      25208.690253229146,
      25390.047737065328,
      25571.40522090151,
      25752.762704737692,
      25934.12018857387,
      26115.477672410052,
      26296.83515624623,
      26478.19264008241,
      26659.55012391859,
      26840.907607754772,
      27022.265091590954,
      27203.622575427133,
      27384.980059263315,
      27566.337543099497,
      27747.69502693568,
      27929.052510771857,
      28110.40999460804,
      28291.76747844422,
      28473.124962280403,
      28654.48244611658,
      28835.839929952763,
      29017.197413788945,
      29198.554897625127,
      29379.912381461305,
      29561.269865297487,
      29742.62734913367,
      29923.98483296985,
      30105.342316806033,
      30286.69980064221,
      30468.057284478393,
      30649.414768314575,
      30830.772252150757,
      31012.129735986935,
      31193.487219823117,
      31374.8447036593,
      31556.20218749548,
      31737.55967133166,
      31918.917155167837,
      32100.27463900402,
      32281.632122840198,
      32462.98960667638,
      32644.34709051256,
      32825.70457434874,
      33007.06205818492,
      33188.4195420211,
      33369.777025857285,
      33551.13450969347,
      33732.49199352965,
      33913.84947736583,
      34095.206961202006,
      34276.56444503819,
      34457.92192887437,
      34639.27941271055,
      34820.63689654673,
      35001.994380382916,
      35183.3518642191,
      35364.70934805528,
      35546.06683189146,
      35727.424315727636,
      35908.78179956382,
      36090.1392834
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0,
      13440000.0,
      14080000.0,
      14720000.0,
      15360000.0,
      16000000.0,
      16640000.0,
      17280000.0,
      17920000.0,
      18560000.0,
      19200000.0,
      19840000.0,
      20480000.0,
      21120000.0,
      21760000.0,
      22400000.0,
      23040000.0,
      23680000.0,
      24320000.0,
      24960000.0,
      25600000.0,
      26240000.0,
      26880000.0,
      27520000.0,
      28160000.0,
      28800000.0,
      29440000.0,
      30080000.0,
      30720000.0,
      31360000.0,
      32000000.0,
      32640000.0,
      33280000.0,
      33920000.0,
      34560000.0,
      35200000.0,
      35840000.0,
      36480000.0,
      37120000.0,
      37760000.0,
      38400000.0,
      39040000.0,
      39680000.0,
      40320000.0,
      40960000.0,
      41600000.0,
      42240000.0,
      42880000.0,
      43520000.0,
      44160000.0,
      44800000.0,
      45440000.0,
      46080000.0,
      46720000.0,
      47360000.0,
      48000000.0,
      48640000.0,
      49280000.0,
      49920000.0,
      50560000.0,
      51200000.0,
      51840000.0,
      52480000.0,
      53120000.0,
      53760000.0,
      54400000.0,
      55040000.0,
      55680000.0,
      56320000.0,
      56960000.0,
      57600000.0,
      58240000.0,
      58880000.0,
      59520000.0,
      60160000.0,
      60800000.0,
      61440000.0,
      62080000.0,
      62720000.0,
      63360000.0,
      64000000.0,
      64640000.0,
      65280000.0,
      65920000.0,
      66560000.0,
      67200000.0,
      67840000.0,
      68480000.0,
      69120000.0,
      69760000.0,
      70400000.0,
      71040000.0,
      71680000.0,
      72320000.0,
      72960000.0,
      73600000.0,
      74240000.0,
      74880000.0,
      75520000.0,
      76160000.0,
      76800000.0,
      77440000.0,
      78080000.0,
      78720000.0,
      79360000.0,
      80000000.0,
      80640000.0,
      81280000.0,
      81920000.0,
      82560000.0,
      83200000.0,
      83840000.0,
      84480000.0,
      85120000.0,
      85760000.0,
      86400000.0,
      87040000.0,
      87680000.0,
      88320000.0,
      88960000.0,
      89600000.0,
      90240000.0,
      90880000.0,
      91520000.0,
      92160000.0,
      92800000.0,
      93440000.0,
      94080000.0,
      94720000.0,
      95360000.0,
      96000000.0,
      96640000.0,
      97280000.0,
      97920000.0,
      98560000.0,
      99200000.0,
      99840000.0,
      100480000.0,
      101120000.0,
      101760000.0,
      102400000.0,
      103040000.0,
      103680000.0,
      104320000.0,
      104960000.0,
      105600000.0,
      106240000.0,
      106880000.0,
      107520000.0,
      108160000.0,
      108800000.0,
      109440000.0,
      110080000.0,
      110720000.0,
      111360000.0,
      112000000.0,
      112640000.0,
      113280000.0,
      113920000.0,
      114560000.0,
      115200000.0,
      115840000.0,
      116480000.0,
      117120000.0,
      117760000.0,
      118400000.0,
      119040000.0,
      119680000.0,
      120320000.0,
      120960000.0,
      121600000.0,
      122240000.0,
      122880000.0,
      123520000.0,
      124160000.0,
      124800000.0,
      125440000.0,
      126080000.0,
      126720000.0,
      127360000.0
    ],
    "return": [
      0.0,
      38.196160634358726,
      58.770947774251304,
      62.07254155476888,
      67.4906005859375,
      70.74937947591145,
      74.7848612467448,
      77.75097147623698,
      79.08344014485677,
      80.12428538004558,
      84.40036265055339,
      84.25612894694011,
      89.841064453125,
      86.2732442220052,
      92.54730733235677,
      95.28934987386067,
      95.8080342610677,
      97.19317372639973,
      107.09626007080078,
      110.65089670817058,
      114.67183176676433,
      110.9939193725586,
      112.79729715983073,
      113.6125971476237,
      116.34737141927083,
      126.48547871907552,
      133.89915466308594,
      109.39595286051433,
      134.32386779785156,
      115.43318176269531,
      143.85650126139322,
      119.99575297037761,
      132.2745564778646,
      135.0002670288086,
      187.80313873291016,
      159.67255147298178,
      146.672732035319,
      158.2981389363607,
      136.65037282307944,
      128.09097798665366,
      169.46741231282553,
      161.30030314127603,
      164.15653228759766,
      137.65259297688803,
      136.9373321533203,
      206.19168599446616,
      159.12822469075522,
      179.7809041341146,
      172.49272664388022,
      250.76256306966147,
      194.66214497884116,
      195.08185323079428,
      235.58951314290366,
      166.8591283162435,
      238.91442362467447,
      245.55697631835938,
      207.1026153564453,
      255.12977600097656,
      217.94174194335938,
      166.34827168782553,
      149.34825642903647,
      183.07278696695963,
      218.01471964518228,
      235.31849161783853,
      195.28186543782553,
      210.81011454264322,
      170.80133565266928,
      211.78978983561197,
      218.24620564778647,
      197.42310587565103,
      182.58748881022134,
      152.4922637939453,
      197.9974594116211,
      201.38032023111978,
      190.21263122558594,
      240.82181294759116,
      286.7570495605469,
      188.37657165527344,
      187.61556498209634,
      203.70641072591147,
      269.580815633138,
      177.3887939453125,
      252.92713419596353,
      267.0267028808594,
      240.3121795654297,
      239.61591593424478,
      207.05109151204428,
      235.18618774414062,
      177.18890889485678,
      202.33190409342447,
      160.5045623779297,
      185.72308349609375,
      188.78704833984375,
      277.3094177246094,
      272.63533782958984,
      224.07214864095053,
      232.78923543294272,
      184.01846822102866,
      225.31009419759116,
      225.0649668375651,
      304.5735168457031,
      210.20008341471353,
      201.0149180094401,
      243.43156941731772,
      215.55621337890625,
      195.11417643229166,
      200.63154093424478,
      250.17939249674478,
      166.86117553710938,
      239.04779052734375,
      202.9445546468099,
      230.7897491455078,
      200.33321634928384,
      166.21000162760416,
      196.61333719889322,
      262.797119140625,
      172.99969991048178,
      223.8022206624349,
      207.91385396321616,
      194.3153839111328,
      213.40963745117188,
      237.738525390625,
      221.23444112141928,
      197.73790486653647,
      180.7091064453125,
      154.77271016438803,
      241.7831827799479,
      224.94592793782553,
      185.94915771484375,
      201.07649739583334,
      219.7017618815104,
      167.22635904947916,
      170.35607401529947,
      163.76718393961588,
      225.6471710205078,
      183.0110321044922,
      192.79330444335938,
      243.40941365559897,
      207.83230590820312,
      187.21705118815103,
      254.02437337239584,
      182.39125061035156,
      186.51834615071616,
      198.46961466471353,
      173.48248799641928,
      169.77811686197916,
      194.39813232421875,
      229.33489481608072,
      197.5059356689453,
      195.6003621419271,
      193.1749471028646,
      201.3477986653646,
      149.3585459391276,
      160.55817159016928,
      262.8863830566406,
      194.60286458333334,
      155.88604227701822,
      155.73231506347656,
      185.68701171875,
      232.66948445638022,
      191.00763956705728,
      184.53072102864584,
      238.246337890625,
      183.43255615234375,
      208.6761220296224,
      198.46814473470053,
      182.61422729492188,
      219.0441640218099,
      272.5106150309245,
      228.05343627929688,
      161.55077107747397,
      202.0955810546875,
      226.70318094889322,
      252.42306518554688,
      248.66568501790366,
      242.6952362060547,
      189.46682739257812,
      151.86629740397134,
      180.39802042643228,
      214.47354125976562,
      190.78692626953125,
      205.54802958170572,
      258.9932454427083,
      253.63004557291666,
      211.98404439290366,
      211.90413411458334,
      191.16903686523438,
      219.9814249674479,
      173.84603881835938,
      207.7472915649414,
      268.8358154296875,
      219.39684041341147,
      195.6284434000651,
      185.613037109375,
      184.69058227539062,
      175.6501668294271,
      221.70868937174478,
      179.950927734375,
      196.78107706705728,
      243.99713134765625
    ],
    "return_min": [
      0.0,
      34.45428217803158,
      55.41737184120804,
      61.41869831871158,
      66.43249621420325,
      69.62003501316896,
      73.62000081838235,
      74.60206213497288,
      76.32229807409914,
      77.34785807783203,
      81.51511554114175,
      79.76769534189077,
      85.01361415766722,
      81.90230639389048,
      89.69955042492953,
      90.6178239151312,
      88.02047685300616,
      96.07391744623853,
      90.61242490649848,
      101.45223290545653,
      95.02559927014411,
      100.60899040578748,
      95.09311835848013,
      97.23245698211139,
      94.93065587379392,
      96.36739335426385,
      105.43736833701234,
      97.00441945236639,
      98.35959134250493,
      98.28940804777696,
      92.67936441640799,
      96.88782462350221,
      104.22682035772064,
      118.07620764500636,
      122.03861283666923,
      77.70721064239504,
      123.05194666169182,
      90.49265642797565,
      109.53681253072628,
      100.79331677933696,
      132.82121169040914,
      118.10119067918774,
      128.9584653469691,
      130.8071722988806,
      134.49261515372942,
      92.08135912203397,
      124.19046377359581,
      98.81203426788495,
      112.60570730312568,
      191.6848021718945,
      169.2558995325841,
      164.44067226472907,
      162.58134839465347,
      101.12928192118886,
      150.588229166145,
      186.23761923215693,
      150.1613026052525,
      164.03645279537557,
      155.29520585208192,
      156.8134730705425,
      110.8007256006957,
      89.45774057195948,
      183.66333080851112,
      182.8045774261979,
      147.439039041815,
      192.50448931520535,
      153.81440899229477,
      159.37387265718547,
      156.8847591896903,
      140.9422189870913,
      133.79386046844553,
      121.2208783359017,
      117.78878466109666,
      162.0891325977859,
      142.91152152327456,
      145.66650485354245,
      181.36210061214183,
      172.0549260337872,
      129.77998980595356,
      184.15597889815282,
      226.4602045321107,
      139.90753198856368,
      212.27614858542546,
      258.7678969036728,
      128.43190294932754,
      155.87710678035796,
      124.28137596113937,
      212.97156265828514,
      144.41096078962323,
      124.89275940718964,
      141.10913428610715,
      166.79807120535114,
      182.2678255457014,
      174.8131193671947,
      117.43064036520028,
      186.94050341879202,
      159.0381560034768,
      155.60826522621846,
      183.84612565643005,
      137.57668363877298,
      271.9786662677094,
      160.17609619913318,
      146.13779258497067,
      175.88971848479008,
      173.54095007784315,
      134.59203053917872,
      166.65528346229607,
      180.4588111039666,
      145.9365899929819,
      151.56289219435445,
      133.81831701051803,
      191.35533174986944,
      157.67126588854296,
      130.87317243361662,
      178.3746080438651,
      229.5139109960802,
      149.1997992438169,
      160.45883006247314,
      160.1759590836749,
      125.17163497894785,
      175.2298130949002,
      187.0507229628084,
      199.16419394083135,
      141.84808456873083,
      120.67866534173005,
      141.746042432658,
      220.1337018131785,
      158.57603198020888,
      148.14785461734908,
      184.8981647429553,
      148.06760980619828,
      122.53421916003592,
      130.70419158023753,
      119.64487618156373,
      154.3790765581566,
      152.53780110145092,
      178.43447367032533,
      194.91483762218263,
      149.96549429029432,
      154.4239946844543,
      169.34695408115252,
      175.13164904117792,
      134.66237652341292,
      172.8040557403638,
      153.92361892495038,
      132.77755809968215,
      158.73452883959555,
      178.48066951373008,
      158.17790159002567,
      184.34224660737877,
      151.50959380362832,
      181.05786787718358,
      128.44826901669964,
      135.85806372404804,
      224.19958239539173,
      159.54753154225168,
      139.97302598768363,
      136.08539724634934,
      169.96330045285345,
      214.40018491853647,
      138.97055888555082,
      174.08251732565873,
      188.24750174073284,
      154.5361398840479,
      147.65977634084902,
      150.8830394081585,
      148.54773711493283,
      195.06914279689272,
      249.31286395949382,
      148.8014394989193,
      128.945207717944,
      171.02724765762608,
      164.33448317784035,
      173.62293096123312,
      211.63861571735728,
      163.8461708101563,
      170.99422189105155,
      127.6976483325336,
      166.47019281749013,
      182.54807929613827,
      124.39573212425708,
      162.95733679973253,
      218.6751066764981,
      200.5109498309493,
      169.39523590474533,
      139.30340693319937,
      164.06790645736223,
      182.94378495233704,
      142.95961881523903,
      146.49638746199017,
      233.57057465778408,
      165.27324945760324,
      171.61117351294516,
      142.4697787158965,
      160.95787142437774,
      163.002228594134,
      167.43590048100222,
      160.83785550209495,
      151.80602190528384,
      205.61825791496244
    ],
    "return_max": [
      0.0,
      41.93803909068587,
      62.12452370729457,
      62.72638479082619,
      68.54870495767175,
      71.87872393865395,
      75.94972167510724,
      80.89988081750109,
      81.8445822156144,
      82.90071268225913,
      87.28560975996503,
      88.74456255198945,
      94.66851474858278,
      90.64418205011992,
      95.395064239784,
      99.96087583259015,
      103.59559166912925,
      98.31243000656094,
      123.58009523510309,
      119.84956051088463,
      134.31806426338454,
      121.3788483393297,
      130.50147596118134,
      129.99273731313602,
      137.76408696474772,
      156.6035640838872,
      162.36094098915953,
      121.78748626866226,
      170.2881442531982,
      132.57695547761367,
      195.03363810637845,
      143.103681317253,
      160.32229259800854,
      151.92432641261084,
      253.56766462915107,
      241.63789230356852,
      170.29351740894617,
      226.10362144474573,
      163.7639331154326,
      155.38863919397033,
      206.11361293524192,
      204.49941560336433,
      199.35459922822622,
      144.49801365489546,
      139.3820491529112,
      320.30201286689834,
      194.06598560791463,
      260.74977400034425,
      232.37974598463475,
      309.84032396742845,
      220.06839042509822,
      225.7230341968595,
      308.5976778911538,
      232.58897471129814,
      327.2406180832039,
      304.8763334045618,
      264.0439281076381,
      346.22309920657756,
      280.5882780346368,
      175.88307030510856,
      187.89578725737724,
      276.68783336195975,
      252.36610848185344,
      287.83240580947916,
      243.12469183383607,
      229.1157397700811,
      187.7882623130438,
      264.20570701403847,
      279.60765210588266,
      253.90399276421076,
      231.38111715199716,
      183.7636492519889,
      278.20613416214553,
      240.67150786445367,
      237.51374092789732,
      335.97712104163986,
      392.1519985089519,
      204.69821727675966,
      245.45114015823913,
      223.2568425536701,
      312.7014267341653,
      214.87005590206132,
      293.5781198065016,
      275.28550885804594,
      352.19245618153184,
      323.3547250881316,
      289.8208070629492,
      257.4008128299961,
      209.96685700009033,
      279.7710487796593,
      179.89999046975223,
      204.64809578683636,
      195.3062711339861,
      379.80571608202405,
      427.84003529397944,
      261.203793863109,
      306.54031486240865,
      212.42867121583885,
      266.77406273875226,
      312.5532500363572,
      337.16836742369685,
      260.2240706302939,
      255.89204343390952,
      310.97342034984536,
      257.57147667996935,
      255.6363223254046,
      234.6077984061935,
      319.8999738895229,
      187.78576108123684,
      326.53268886033305,
      272.0707922831018,
      270.2241665411462,
      242.99516681002473,
      201.5468308215917,
      214.85206635392134,
      296.0803272851698,
      196.79960057714666,
      287.1456112623967,
      255.6517488427574,
      263.4591328433178,
      251.58946180744354,
      288.42632781844156,
      243.3046883020072,
      253.6277251643421,
      240.73954754889496,
      167.79937789611807,
      263.4326637467173,
      291.3158238954422,
      223.75046081233842,
      217.2548300487114,
      291.33591395682254,
      211.9184989389224,
      210.0079564503614,
      207.889491697668,
      296.915265482859,
      213.48426310753345,
      207.15213521639342,
      291.9039896890153,
      265.69911752611193,
      220.01010769184776,
      338.70179266363914,
      189.6508521795252,
      238.3743157780194,
      224.13517358906327,
      193.04135706788819,
      206.77867562427616,
      230.06173580884195,
      280.18912011843133,
      236.83396974786496,
      206.85847767647542,
      234.84030040210087,
      221.6377294535456,
      170.26882286155555,
      185.25827945629052,
      301.5731837178895,
      229.658197624415,
      171.7990585663528,
      175.37923288060378,
      201.41072298464655,
      250.93878399422397,
      243.04472024856375,
      194.97892473163296,
      288.24517404051716,
      212.3289724206396,
      269.6924677183958,
      246.05325006124255,
      216.68071747491092,
      243.0191852467271,
      295.70836610235517,
      307.30543305967444,
      194.15633443700395,
      233.16391445174892,
      289.0718787199461,
      331.2231994098606,
      285.69275431845006,
      321.54430160195307,
      207.9394328941047,
      176.0349464754091,
      194.32584803537443,
      246.39900322339298,
      257.17812041480545,
      248.1387223636789,
      299.3113842089185,
      306.749141314884,
      254.57285288106198,
      284.5048612959673,
      218.27016727310652,
      257.0190649825588,
      204.73245882147972,
      268.9981956678926,
      304.10105620159095,
      273.5204313692197,
      219.64571328718503,
      228.7562955028535,
      208.4232931264035,
      188.29810506472018,
      275.98147826248737,
      199.06399996665505,
      241.75613222883072,
      282.37600478035006
    ]
  },
  "h1hand_stair": {
    "time": [
      0.0,
      387.40268051806663,
      774.8053610361333,
      1162.2080415542,
      1549.6107220722665,
      1937.013402590333,
      2324.4160831084,
      2711.818763626466,
      3099.221444144533,
      3486.6241246626,
      3874.026805180666,
      4261.4294856987335,
      4648.8321662168,
      5036.234846734867,
      5423.637527252932,
      5811.040207771,
      6198.442888289066,
      6585.845568807133,
      6973.2482493252,
      7360.650929843266,
      7748.053610361332,
      8135.4562908794,
      8522.858971397467,
      8910.261651915533,
      9297.6643324336,
      9685.067012951667,
      10072.469693469733,
      10459.872373987799,
      10847.275054505864,
      11234.677735023932,
      11622.080415542
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0,
      13440000.0,
      14080000.0,
      14720000.0,
      15360000.0,
      16000000.0,
      16640000.0,
      17280000.0,
      17920000.0,
      18560000.0,
      19200000.0
    ],
    "return": [
      0.0,
      23.89271863301595,
      34.71700096130371,
      60.09582392374674,
      73.17438507080078,
      151.34715779622397,
      224.9233856201172,
      271.56304423014325,
      316.3719889322917,
      371.5347391764323,
      382.11834716796875,
      427.6658426920573,
      422.29393513997394,
      487.94639078776044,
      474.13771565755206,
      548.6403503417969,
      592.8809407552084,
      625.5245564778646,
      660.0660400390625,
      652.6300048828125,
      654.2977498372396,
      667.5510660807291,
      689.77783203125,
      690.3555501302084,
      692.7436726888021,
      714.7565511067709,
      700.9408976236979,
      730.7681477864584,
      744.6813557942709,
      703.8641560872396,
      689.6199951171875
    ],
    "return_min": [
      0.0,
      15.890752302265494,
      29.305759036971846,
      45.07623410366991,
      64.0256023149527,
      108.29035638002946,
      168.69137872905253,
      235.03404534904712,
      268.15860258395946,
      351.4744388725385,
      372.16143158111464,
      411.6028455251883,
      342.44605901687726,
      425.6015045319289,
      438.91894798410647,
      466.1055596018814,
      559.5168597506888,
      596.1932083138497,
      610.9713671046555,
      592.2205757048379,
      609.8229299953937,
      623.5360991628968,
      650.7099879959824,
      643.435216307602,
      640.6507078943843,
      684.2973876188141,
      674.313749484506,
      702.2090532268096,
      720.8903004478105,
      644.9576463371507,
      627.955726274373
    ],
    "return_max": [
      0.0,
      31.894684963766405,
      40.12824288563557,
      75.11541374382357,
      82.32316782664886,
      194.40395921241847,
      281.15539251118184,
      308.0920431112394,
      364.5853752806239,
      391.59503948032614,
      392.07526275482286,
      443.72883985892634,
      502.1418112630706,
      550.2912770435919,
      509.35648333099766,
      631.1751410817124,
      626.2450217597279,
      654.8559046418795,
      709.1607129734695,
      713.0394340607871,
      698.7725696790856,
      711.5660329985615,
      728.8456760665176,
      737.2758839528148,
      744.83663748322,
      745.2157145947276,
      727.5680457628897,
      759.3272423461071,
      768.4724111407312,
      762.7706658373286,
      751.284263960002
    ]
  },
  "h1hand_slide": {
    "time": [
      0.0,
      309.2993865171333,
      618.5987730342666,
      927.8981595514,
      1237.1975460685333,
      1546.4969325856666,
      1855.7963191028,
      2165.0957056199336,
      2474.3950921370665,
      2783.6944786542,
      3092.9938651713333,
      3402.293251688467,
      3711.5926382056,
      4020.8920247227334,
      4330.191411239867,
      4639.490797757,
      4948.790184274133,
      5258.089570791267,
      5567.3889573084,
      5876.688343825534,
      6185.9877303426665,
      6495.2871168598,
      6804.586503376934,
      7113.885889894067,
      7423.1852764112,
      7732.484662928333,
      8041.784049445467,
      8351.0834359626,
      8660.382822479734,
      8969.682208996868,
      9278.981595514
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0,
      13440000.0,
      14080000.0,
      14720000.0,
      15360000.0,
      16000000.0,
      16640000.0,
      17280000.0,
      17920000.0,
      18560000.0,
      19200000.0
    ],
    "return": [
      0.0,
      25.91161886850993,
      46.50613911946615,
      69.50710932413737,
      125.7647476196289,
      170.76056416829428,
      238.53360493977866,
      265.1405283610026,
      287.49407958984375,
      301.4013264973958,
      266.9022521972656,
      286.79445393880206,
      311.7269694010417,
      334.2634684244792,
      393.5237731933594,
      413.90247599283856,
      468.1215108235677,
      478.6412353515625,
      521.7706604003906,
      597.3247273763021,
      682.8544311523438,
      815.7204182942709,
      869.786865234375,
      870.6855061848959,
      924.8785807291666,
      915.4677937825521,
      915.6895751953125,
      901.403076171875,
      910.4904378255209,
      935.8196411132812,
      904.0150146484375
    ],
    "return_min": [
      0.0,
      17.31227752983613,
      41.81182919734604,
      57.14489937214505,
      96.5149593569322,
      163.1428876300595,
      222.31254684790358,
      231.41575879824072,
      255.4749652490164,
      243.84204384859126,
      249.15111161639513,
      265.7686367908954,
      283.11169371305834,
      310.17930949833135,
      345.48766050051773,
      399.79618209856903,
      452.1210687697244,
      462.9765296109677,
      495.99326194394064,
      558.9139653990013,
      668.4022813113013,
      795.2447221786368,
      830.8790856292156,
      823.1121698948539,
      915.6907611174105,
      893.0342590385319,
      869.1971213119441,
      878.6902612410832,
      894.2164153448738,
      921.1641786848095,
      859.1733242686221
    ],
    "return_max": [
      0.0,
      34.51096020718373,
      51.20044904158625,
      81.8693192761297,
      155.01453588232562,
      178.37824070652906,
      254.75466303165373,
      298.86529792376456,
      319.5131939306711,
      358.9606091462004,
      284.6533927781361,
      307.8202710867087,
      340.342245089025,
      358.347627350627,
      441.559885886201,
      428.0087698871081,
      484.121952877411,
      494.3059410921573,
      547.5480588568406,
      635.7354893536029,
      697.3065809933862,
      836.1961144099049,
      908.6946448395344,
      918.2588424749379,
      934.0664003409228,
      937.9013285265723,
      962.1820290786809,
      924.1158911026668,
      926.764460306168,
      950.475103541753,
      948.8567050282529
    ]
  },
  "h1hand_pole": {
    "time": [
      0.0,
      385.46620238036667,
      770.9324047607333,
      1156.3986071411002,
      1541.8648095214667,
      1927.331011901833,
      2312.7972142822005,
      2698.2634166625667,
      3083.7296190429333,
      3469.1958214233,
      3854.662023803666,
      4240.128226184033,
      4625.594428564401,
      5011.060630944767,
      5396.526833325133,
      5781.9930357055,
      6167.459238085867,
      6552.925440466233,
      6938.3916428466,
      7323.857845226967,
      7709.324047607332,
      8094.7902499877,
      8480.256452368067,
      8865.722654748433,
      9251.188857128802,
      9636.655059509167,
      10022.121261889533,
      10407.5874642699,
      10793.053666650267,
      11178.519869030633,
      11563.986071411
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0,
      13440000.0,
      14080000.0,
      14720000.0,
      15360000.0,
      16000000.0,
      16640000.0,
      17280000.0,
      17920000.0,
      18560000.0,
      19200000.0
    ],
    "return": [
      0.0,
      86.94035212198894,
      315.2128092447917,
      259.81410725911456,
      438.83277384440106,
      453.3154805501302,
      472.5972188313802,
      600.6257120768229,
      602.0543416341146,
      604.5667724609375,
      625.5578002929688,
      697.9258626302084,
      730.2250773111979,
      770.3667805989584,
      756.2709757486979,
      817.2851765950521,
      814.6104939778646,
      834.628662109375,
      836.6917317708334,
      836.28759765625,
      802.5843302408854,
      828.5123494466146,
      889.4725545247396,
      927.0976359049479,
      918.7555135091146,
      918.3529663085938,
      929.7622884114584,
      927.1256306966146,
      924.934814453125,
      935.3818562825521,
      867.1444905598959
    ],
    "return_min": [
      0.0,
      34.151917472845525,
      169.25709369786907,
      195.91231211958,
      309.209006533762,
      333.5828818070501,
      457.3834555761806,
      538.0321487054368,
      556.7912159567086,
      546.9315820056133,
      562.6627573317008,
      625.2555533079764,
      698.4326425070107,
      708.3732902267687,
      687.8694938402527,
      787.8280779660413,
      789.6780869058325,
      772.5260295300255,
      792.1538079552,
      783.306104588348,
      739.7921613627637,
      715.3343740103686,
      847.4289290942377,
      887.3489508402633,
      877.1328123681296,
      863.362262954571,
      880.872949690255,
      877.8235128123039,
      882.9430126525199,
      900.4706760458522,
      743.7438399173179
    ],
    "return_max": [
      0.0,
      139.72878677113235,
      461.1685247917143,
      323.7159023986491,
      568.4565411550401,
      573.0480792932102,
      487.81098208657977,
      663.2192754482089,
      647.3174673115207,
      662.2019629162617,
      688.4528432542367,
      770.5961719524404,
      762.0175121153851,
      832.360270971148,
      824.672457657143,
      846.742275224063,
      839.5429010498967,
      896.7312946887245,
      881.2296555864667,
      889.269090724152,
      865.376499119007,
      941.6903248828606,
      931.5161799552416,
      966.8463209696324,
      960.3782146500996,
      973.3436696626165,
      978.6516271326617,
      976.4277485809254,
      966.9266162537301,
      970.293036519252,
      990.5451412024738
    ]
  },
  "h1hand_push": {
    "time": [
      0.0,
      312.6167574498032,
      625.2335148996063,
      937.8502723494095,
      1250.4670297992127,
      1563.083787249016,
      1875.700544698819,
      2188.3173021486223,
      2500.9340595984254,
      2813.5508170482285,
      3126.167574498032,
      3438.784331947835,
      3751.401089397638,
      4064.0178468474414,
      4376.634604297245,
      4689.251361747049,
      5001.868119196851,
      5314.484876646654,
      5627.101634096457,
      5939.718391546261,
      6252.335148996064,
      6564.951906445867,
      6877.56866389567,
      7190.1854213454735,
      7502.802178795276,
      7815.418936245081,
      8128.035693694883,
      8440.652451144686,
      8753.26920859449,
      9065.885966044292,
      9378.502723494097,
      9691.119480943898,
      10003.736238393702,
      10316.352995843505,
      10628.969753293308,
      10941.586510743113,
      11254.203268192914,
      11566.820025642717,
      11879.436783092522,
      12192.053540542325,
      12504.670297992128,
      12817.287055441931,
      13129.903812891735,
      13442.520570341538,
      13755.13732779134,
      14067.754085241144,
      14380.370842690947,
      14692.987600140748,
      15005.604357590551,
      15318.221115040358,
      15630.837872490161,
      15943.454629939963,
      16256.071387389766,
      16568.68814483957,
      16881.304902289372,
      17193.921659739175,
      17506.53841718898,
      17819.15517463878,
      18131.771932088584,
      18444.38868953839,
      18757.005446988194,
      19069.622204437994,
      19382.238961887797,
      19694.8557193376,
      20007.472476787403,
      20320.089234237206,
      20632.70599168701,
      20945.322749136812,
      21257.939506586616,
      21570.55626403642,
      21883.173021486225,
      22195.78977893603,
      22508.406536385828,
      22821.02329383563,
      23133.640051285434,
      23446.25680873524,
      23758.873566185044,
      24071.490323634847,
      24384.10708108465,
      24696.723838534454,
      25009.340595984257,
      25321.95735343406,
      25634.574110883863,
      25947.190868333666,
      26259.80762578347,
      26572.424383233272,
      26885.041140683075,
      27197.65789813288,
      27510.27465558268,
      27822.891413032485,
      28135.508170482288,
      28448.12492793209,
      28760.741685381894,
      29073.358442831694,
      29385.975200281497,
      29698.5919577313,
      30011.208715181103,
      30323.825472630913,
      30636.442230080716,
      30949.05898753052,
      31261.675744980323,
      31574.292502430126,
      31886.909259879925,
      32199.52601732973,
      32512.14277477953,
      32824.759532229335,
      33137.37628967914,
      33449.99304712894,
      33762.609804578744,
      34075.22656202855,
      34387.84331947835,
      34700.46007692815,
      35013.07683437796,
      35325.69359182776,
      35638.31034927756,
      35950.927106727366,
      36263.54386417717,
      36576.16062162697,
      36888.77737907678,
      37201.394136526585,
      37514.01089397639,
      37826.62765142619,
      38139.24440887599,
      38451.86116632579,
      38764.477923775594,
      39077.0946812254,
      39389.7114386752,
      39702.328196125,
      40014.944953574806,
      40327.56171102461,
      40640.17846847441,
      40952.795225924216,
      41265.41198337402,
      41578.02874082382,
      41890.645498273625,
      42203.26225572343,
      42515.87901317323,
      42828.495770623034,
      43141.11252807284,
      43453.72928552265,
      43766.34604297245,
      44078.962800422254,
      44391.57955787206,
      44704.19631532185,
      45016.813072771656,
      45329.42983022146,
      45642.04658767126,
      45954.663345121066,
      46267.28010257087,
      46579.89686002067,
      46892.51361747048,
      47205.130374920285,
      47517.74713237009,
      47830.36388981989,
      48142.980647269695,
      48455.5974047195,
      48768.2141621693,
      49080.830919619104,
      49393.44767706891,
      49706.06443451871,
      50018.68119196851,
      50331.29794941832,
      50643.91470686812,
      50956.53146431792,
      51269.148221767726,
      51581.76497921753,
      51894.38173666733,
      52206.998494117135,
      52519.61525156694,
      52832.23200901674,
      53144.848766466544,
      53457.46552391635,
      53770.08228136615,
      54082.699038815954,
      54395.31579626576,
      54707.93255371556,
      55020.54931116536,
      55333.166068615166,
      55645.78282606497,
      55958.39958351477,
      56271.016340964576,
      56583.63309841438,
      56896.24985586418,
      57208.866613313985,
      57521.48337076379,
      57834.10012821359,
      58146.71688566339,
      58459.33364311319,
      58771.95040056299
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0,
      13440000.0,
      14080000.0,
      14720000.0,
      15360000.0,
      16000000.0,
      16640000.0,
      17280000.0,
      17920000.0,
      18560000.0,
      19200000.0,
      19840000.0,
      20480000.0,
      21120000.0,
      21760000.0,
      22400000.0,
      23040000.0,
      23680000.0,
      24320000.0,
      24960000.0,
      25600000.0,
      26240000.0,
      26880000.0,
      27520000.0,
      28160000.0,
      28800000.0,
      29440000.0,
      30080000.0,
      30720000.0,
      31360000.0,
      32000000.0,
      32640000.0,
      33280000.0,
      33920000.0,
      34560000.0,
      35200000.0,
      35840000.0,
      36480000.0,
      37120000.0,
      37760000.0,
      38400000.0,
      39040000.0,
      39680000.0,
      40320000.0,
      40960000.0,
      41600000.0,
      42240000.0,
      42880000.0,
      43520000.0,
      44160000.0,
      44800000.0,
      45440000.0,
      46080000.0,
      46720000.0,
      47360000.0,
      48000000.0,
      48640000.0,
      49280000.0,
      49920000.0,
      50560000.0,
      51200000.0,
      51840000.0,
      52480000.0,
      53120000.0,
      53760000.0,
      54400000.0,
      55040000.0,
      55680000.0,
      56320000.0,
      56960000.0,
      57600000.0,
      58240000.0,
      58880000.0,
      59520000.0,
      60160000.0,
      60800000.0,
      61440000.0,
      62080000.0,
      62720000.0,
      63360000.0,
      64000000.0,
      64640000.0,
      65280000.0,
      65920000.0,
      66560000.0,
      67200000.0,
      67840000.0,
      68480000.0,
      69120000.0,
      69760000.0,
      70400000.0,
      71040000.0,
      71680000.0,
      72320000.0,
      72960000.0,
      73600000.0,
      74240000.0,
      74880000.0,
      75520000.0,
      76160000.0,
      76800000.0,
      77440000.0,
      78080000.0,
      78720000.0,
      79360000.0,
      80000000.0,
      80640000.0,
      81280000.0,
      81920000.0,
      82560000.0,
      83200000.0,
      83840000.0,
      84480000.0,
      85120000.0,
      85760000.0,
      86400000.0,
      87040000.0,
      87680000.0,
      88320000.0,
      88960000.0,
      89600000.0,
      90240000.0,
      90880000.0,
      91520000.0,
      92160000.0,
      92800000.0,
      93440000.0,
      94080000.0,
      94720000.0,
      95360000.0,
      96000000.0,
      96640000.0,
      97280000.0,
      97920000.0,
      98560000.0,
      99200000.0,
      99840000.0,
      100480000.0,
      101120000.0,
      101760000.0,
      102400000.0,
      103040000.0,
      103680000.0,
      104320000.0,
      104960000.0,
      105600000.0,
      106240000.0,
      106880000.0,
      107520000.0,
      108160000.0,
      108800000.0,
      109440000.0,
      110080000.0,
      110720000.0,
      111360000.0,
      112000000.0,
      112640000.0,
      113280000.0,
      113920000.0,
      114560000.0,
      115200000.0,
      115840000.0,
      116480000.0,
      117120000.0,
      117760000.0,
      118400000.0,
      119040000.0,
      119680000.0,
      120320000.0
    ],
    "return": [
      0.0,
      -465.70371500651044,
      -209.66437276204428,
      -215.35665893554688,
      -79.81609725952148,
      -97.29651769002278,
      -171.10615030924478,
      -58.52375920613607,
      35.39680035909017,
      67.48361078898112,
      17.984004974365234,
      46.02268600463867,
      106.70371754964192,
      52.096457163492836,
      87.57223892211914,
      60.093360900878906,
      202.40312703450522,
      35.78864034016927,
      77.78755187988281,
      199.13873799641928,
      161.0906524658203,
      -6.331198374430339,
      227.29449462890625,
      225.0338363647461,
      311.14427693684894,
      233.39718119303384,
      389.0839029947917,
      355.971435546875,
      262.3876698811849,
      351.2383626302083,
      315.8093973795573,
      400.12449137369794,
      398.5304260253906,
      408.3233947753906,
      473.21187082926434,
      377.5498504638672,
      485.5910339355469,
      400.6451822916667,
      458.1425069173177,
      439.06847127278644,
      326.82744852701825,
      403.84735616048175,
      480.24944559733075,
      495.53029378255206,
      483.7826232910156,
      533.0876261393229,
      472.01613362630206,
      496.1668395996094,
      512.2647094726562,
      514.5851847330729,
      470.1879475911458,
      544.6070658365885,
      360.79221598307294,
      496.0656331380208,
      504.4897104899089,
      546.812978108724,
      533.8811543782552,
      535.4517415364584,
      471.8639500935872,
      557.7514444986979,
      629.8241882324219,
      534.7682189941406,
      511.1558583577474,
      564.1106160481771,
      494.4066518147786,
      652.6817423502604,
      521.490966796875,
      520.7968419392904,
      562.0110270182291,
      456.7043991088867,
      605.8467203776041,
      640.2967529296875,
      603.9107055664062,
      574.7682291666666,
      611.0949910481771,
      631.2831217447916,
      605.4075927734375,
      546.4880421956381,
      651.6212768554688,
      625.2835998535156,
      702.7826131184896,
      600.3868408203125,
      652.3099568684896,
      490.0066731770833,
      593.0624084472656,
      541.475596110026,
      587.3904622395834,
      612.0767822265625,
      577.2430826822916,
      571.9033610026041,
      625.8723347981771,
      488.1465759277344,
      592.8579711914062,
      510.37445576985675,
      523.4727376302084,
      603.7941080729166,
      547.8638305664062,
      583.2688598632812,
      570.062510172526,
      527.5540873209635,
      582.3120727539062,
      656.2349243164062,
      508.43341064453125,
      607.9522298177084,
      621.880859375,
      636.3435770670573,
      665.6842244466146,
      588.9774169921875,
      636.5202026367188,
      662.1762288411459,
      561.2283732096354,
      598.4204813639323,
      546.9343566894531,
      626.6971232096354,
      652.875732421875,
      466.8674825032552,
      524.4137674967448,
      582.6043701171875,
      534.4446411132812,
      614.892588297526,
      673.6151529947916,
      547.5692138671875,
      635.2989908854166,
      611.1135660807291,
      512.3421630859375,
      652.3751627604166,
      667.9252522786459,
      649.40771484375,
      579.6252848307291,
      612.9027201334635,
      615.0734354654948,
      541.9854431152344,
      587.8748372395834,
      592.4884033203125,
      755.3507893880209,
      699.7528279622396,
      717.8043619791666,
      616.4128214518229,
      610.3042602539062,
      576.1018473307291,
      669.9233601888021,
      662.5718587239584,
      601.7003580729166,
      619.9717000325521,
      616.17236328125,
      647.919189453125,
      720.2536214192709,
      676.1502685546875,
      638.7950236002604,
      725.2923787434896,
      708.4577229817709,
      657.550038655599,
      606.4547119140625,
      720.4335225423177,
      638.5685424804688,
      644.5047810872396,
      675.3748372395834,
      545.7748819986979,
      775.0440673828125,
      696.5569864908854,
      684.9332377115885,
      675.4559529622396,
      626.1231486002604,
      670.3891398111979,
      633.9583841959635,
      662.5113728841146,
      625.4878743489584,
      723.1873982747396,
      747.1727294921875,
      709.0388793945312,
      723.6645100911459,
      625.6194051106771,
      593.6909993489584,
      727.9166056315104,
      689.2951049804688,
      630.6175944010416,
      730.9366251627604,
      689.8072102864584,
      696.1072387695312,
      611.1871439615885,
      745.7859090169271,
      636.9237670898438,
      793.2918090820312,
      752.7328694661459,
      767.5932413736979,
      738.7565307617188,
      687.819091796875,
      690.4653930664062,
      769.5697631835938
    ],
    "return_min": [
      0.0,
      -651.7377904873346,
      -218.19490330067313,
      -230.40671483223093,
      -105.68158272497024,
      -210.8956213842916,
      -233.1911752316335,
      -161.58889381026177,
      26.55360975511062,
      4.289192049179803,
      -94.47579688203697,
      4.849184562816511,
      69.07344829711946,
      -22.40429315765919,
      45.086476580251066,
      -54.71491217125336,
      125.38167282396118,
      -98.6124742090824,
      -38.373892937135324,
      145.39119983240533,
      134.49273414913256,
      -59.93287233128325,
      171.49033469732623,
      133.5127446996148,
      206.89851474999222,
      177.69275565837683,
      239.91640177783498,
      218.16699335525163,
      125.28320106593608,
      177.69565455876588,
      91.22711231593178,
      174.67941277233874,
      224.61917253986607,
      176.72156917446412,
      178.20974585184655,
      162.2428865098692,
      308.5236004576717,
      101.43215919483697,
      205.59336156515505,
      218.07057710774632,
      76.95591916101498,
      102.58860226185669,
      256.5846713119894,
      279.8230541236833,
      288.91774124081064,
      285.3080260060623,
      376.76426537108176,
      292.3935583245744,
      229.1453409203956,
      294.52281767361126,
      302.78698861571456,
      356.1194735219518,
      69.45947274408792,
      383.9554921423231,
      298.3401702958506,
      321.4518209660676,
      315.3049143709103,
      346.0651825103665,
      151.19494125741988,
      357.1257353560337,
      390.8240639442132,
      348.42316335307174,
      298.91965969292187,
      462.3853769182599,
      251.6649901787817,
      479.78588898797443,
      397.877011500767,
      215.596613915609,
      323.2564485248071,
      186.6042427741824,
      426.8233101559147,
      362.0302869053308,
      404.98571774036856,
      359.2847078708686,
      484.5054633019436,
      493.2501176605723,
      411.44129488977313,
      256.5276751004272,
      454.29501648725363,
      413.8562263314633,
      573.720797985042,
      416.46692181466364,
      467.0122085781444,
      258.0562843972043,
      395.2243701392048,
      368.5233705526356,
      452.636496904995,
      464.5032990675227,
      436.79629123776,
      374.69121493690363,
      466.7821493928696,
      263.51951290052034,
      429.315411959895,
      304.9247130991566,
      419.90325565387303,
      432.2364118278375,
      338.1089545341084,
      364.2132868278239,
      393.04972714186283,
      359.9071598690796,
      533.942556029381,
      468.2753658460946,
      261.8154182211118,
      522.5536947167411,
      557.584211053626,
      510.90186706032495,
      524.9047980503793,
      487.37465424235245,
      467.86209848079557,
      545.6308974446529,
      377.0755931355628,
      422.04626971453774,
      406.3920283244486,
      444.8626825076674,
      540.525108305069,
      420.5410566062819,
      489.81667849244985,
      441.43925917296247,
      289.6483089231782,
      516.6815783827745,
      591.9147876948792,
      376.4286871094273,
      534.641886307782,
      388.33717493137976,
      322.51834109876495,
      523.6089601382292,
      557.0252517096708,
      537.192194230057,
      479.63953220680753,
      489.9297848053796,
      456.3615203978124,
      452.74782656646664,
      437.8910113628925,
      444.34476831068963,
      622.8334792297339,
      544.8195929605185,
      599.7323369153378,
      512.7902955981406,
      406.1789579431253,
      452.78874036721595,
      582.1919827487237,
      594.584856475092,
      518.5465718482806,
      598.0693065627131,
      472.0681568588566,
      485.4507956994522,
      590.2059351368265,
      586.9153657898868,
      589.259669009053,
      635.5611082804211,
      623.3885305260387,
      452.57285763522157,
      565.053967031291,
      539.3486244195427,
      504.29379132207185,
      558.5882700286743,
      508.762833637914,
      356.62391796920565,
      629.1636777085658,
      556.0355628400487,
      444.2008257553892,
      524.9059378555166,
      404.75813990024164,
      517.9978694633127,
      513.871824831959,
      612.3382451960599,
      516.9922775142898,
      680.363477847471,
      627.5686018024139,
      600.6839979902597,
      614.6357271298568,
      381.38851880783363,
      525.7235252030711,
      636.9343357735295,
      617.0474412446536,
      520.9154402643403,
      551.0778985607511,
      616.0456681349963,
      608.2566427500684,
      506.0818447569322,
      715.3050290539916,
      469.6640181908483,
      710.4375439178509,
      677.2274044637059,
      715.0402021473672,
      669.0276233195295,
      590.3275579897734,
      592.3499843504039,
      654.5575435877125
    ],
    "return_max": [
      0.0,
      -279.6696395256863,
      -201.13384222341543,
      -200.30660303886282,
      -53.95061179407273,
      16.30258600424601,
      -109.02112538685606,
      44.54137539798962,
      44.23999096306972,
      130.67802952878245,
      130.44380683076744,
      87.19618744646084,
      144.3339868021644,
      126.59720748464485,
      130.05800126398722,
      174.90163397301117,
      279.42458124504924,
      170.18975488942095,
      193.94899669690096,
      252.88627616043323,
      187.68857078250807,
      47.27047558242258,
      283.09865456048624,
      316.5549280298774,
      415.39003912370566,
      289.10160672769086,
      538.2514042117484,
      493.77587773849837,
      399.4921386964337,
      524.7810707016507,
      540.3916824431828,
      625.5695699750571,
      572.4416795109152,
      639.9252203763172,
      768.2139958066821,
      592.8568144178652,
      662.658467413422,
      699.8582053884963,
      710.6916522694803,
      660.0663654378266,
      576.6989778930215,
      705.1061100591069,
      703.9142198826721,
      711.2375334414209,
      678.6475053412206,
      780.8672262725835,
      567.2680018815224,
      699.9401208746443,
      795.3840780249169,
      734.6475517925345,
      637.5889065665771,
      733.0946581512252,
      652.1249592220579,
      608.1757741337185,
      710.6392506839671,
      772.1741352513804,
      752.4573943856002,
      724.8383005625502,
      792.5329589297546,
      758.377153641362,
      868.8243125206305,
      721.1132746352096,
      723.3920570225729,
      665.8358551780943,
      737.1483134507755,
      825.5775957125463,
      645.104922092983,
      825.9970699629719,
      800.7656055116511,
      726.804555443591,
      784.8701305992936,
      918.5632189540443,
      802.835693392444,
      790.2517504624647,
      737.6845187944107,
      769.316125829011,
      799.3738906571018,
      836.448409290849,
      848.9475372236839,
      836.710973375568,
      831.8444282519372,
      784.3067598259613,
      837.6077051588348,
      721.9570619569623,
      790.9004467553265,
      714.4278216674164,
      722.1444275741717,
      759.6502653856023,
      717.6898741268233,
      769.1155070683046,
      784.9625202034847,
      712.7736389549484,
      756.4005304229174,
      715.824198440557,
      627.0422196065438,
      775.3518043179957,
      757.6187065987041,
      802.3244328987387,
      747.0752932031892,
      695.2010147728474,
      630.6815894784315,
      844.1944827867179,
      755.0514030679507,
      693.3507649186756,
      686.177507696374,
      761.7852870737895,
      806.4636508428499,
      690.5801797420226,
      805.1783067926419,
      778.7215602376389,
      745.3811532837079,
      774.7946930133268,
      687.4766850544577,
      808.5315639116034,
      765.226356538681,
      513.1939084002285,
      559.0108565010396,
      723.7694810614125,
      779.2409733033843,
      713.1035982122775,
      755.315518294704,
      718.7097406249477,
      735.9560954630513,
      833.8899572300785,
      702.16598507311,
      781.141365382604,
      778.825252847621,
      761.623235457443,
      679.6110374546507,
      735.8756554615474,
      773.785350533177,
      631.2230596640021,
      737.8586631162742,
      740.6320383299353,
      887.8680995463078,
      854.6860629639607,
      835.8763870429955,
      720.0353473055052,
      814.4295625646872,
      699.4149542942423,
      757.6547376288805,
      730.5588609728247,
      684.8541442975527,
      641.8740935023911,
      760.2765697036434,
      810.3875832067978,
      850.3013077017152,
      765.3851713194882,
      688.3303781914677,
      815.0236492065582,
      793.526915437503,
      862.5272196759764,
      647.855456796834,
      901.5184206650928,
      772.8432936388656,
      730.421292145805,
      841.9868408412527,
      734.9258460281901,
      920.9244570570592,
      837.0784101417221,
      925.6656496677879,
      826.0059680689626,
      847.4881573002791,
      822.7804101590831,
      754.044943559968,
      712.6845005721693,
      733.983471183627,
      766.0113187020082,
      866.7768571819611,
      817.3937607988028,
      832.6932930524349,
      869.8502914135206,
      661.6584734948457,
      818.8988754894913,
      761.542768716284,
      740.319748537743,
      910.7953517647696,
      763.5687524379205,
      783.9578347889941,
      716.2924431662449,
      776.2667889798627,
      804.1835159888392,
      876.1460742462116,
      828.2383344685859,
      820.1462806000286,
      808.485438203908,
      785.3106256039766,
      788.5808017824086,
      884.581982779475
    ]
  },
  "h1hand_cabinet": {
    "time": [
      0.0,
      482.0103869639,
      964.0207739278,
      1446.0311608917,
      1928.0415478556,
      2410.0519348195,
      2892.0623217834,
      3374.0727087472997,
      3856.0830957112,
      4338.0934826751,
      4820.103869639,
      5302.114256602899,
      5784.1246435668,
      6266.1350305307,
      6748.145417494599,
      7230.155804458499,
      7712.1661914224,
      8194.1765783863,
      8676.1869653502,
      9158.197352314099,
      9640.207739278
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0
    ],
    "return": [
      0.0,
      48.33211135864258,
      41.21469751993815,
      59.57719930013021,
      75.67262268066406,
      56.34848594665527,
      81.90048472086589,
      80.5737673441569,
      81.59727350870769,
      73.00228627522786,
      88.43795522054036,
      141.72611236572266,
      114.48175557454427,
      148.1658706665039,
      112.07423400878906,
      155.38145955403647,
      130.3720957438151,
      148.42277018229166,
      155.57244873046875,
      131.13333638509116,
      188.2349853515625
    ],
    "return_min": [
      0.0,
      35.922316004492785,
      28.06647614718972,
      37.90235693360684,
      44.75038850207668,
      36.40830307855872,
      61.24509401202533,
      63.87340072139955,
      61.64813090949677,
      32.291284324269796,
      66.49554026294187,
      118.72944532889935,
      74.6534328246156,
      99.11059146542257,
      80.17006584290101,
      130.34182639754522,
      98.22037494029969,
      132.83954440041933,
      144.75589448674302,
      102.16349649420323,
      136.03014741728717
    ],
    "return_max": [
      0.0,
      60.74190671279237,
      54.36291889268658,
      81.25204166665358,
      106.59485685925145,
      76.28866881475183,
      102.55587542970645,
      97.27413396691426,
      101.5464161079186,
      113.71328822618592,
      110.38037017813885,
      164.72277940254597,
      154.31007832447293,
      197.22114986758524,
      143.97840217467711,
      180.42109271052772,
      162.5238165473305,
      164.005995964164,
      166.38900297419448,
      160.10317627597908,
      240.43982328583783
    ]
  },
  "h1hand_door": {
    "time": [
      0.0,
      474.1971419775,
      948.394283955,
      1422.5914259325,
      1896.78856791,
      2370.9857098875,
      2845.182851865,
      3319.3799938425,
      3793.57713582,
      4267.7742777975,
      4741.971419775,
      5216.1685617525,
      5690.36570373,
      6164.5628457075,
      6638.759987685,
      7112.9571296625,
      7587.15427164,
      8061.3514136175,
      8535.548555595,
      9009.7456975725,
      9483.94283955
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0
    ],
    "return": [
      0.0,
      113.92080434163411,
      225.69124348958334,
      249.13692220052084,
      276.68773396809894,
      287.51111857096356,
      287.70652262369794,
      298.76854451497394,
      303.0335388183594,
      305.23622639973956,
      311.20665486653644,
      312.6994222005208,
      316.5781656901042,
      321.4351806640625,
      323.1549580891927,
      328.4767761230469,
      331.2190246582031,
      332.8224283854167,
      333.1351013183594,
      330.49583943684894,
      331.7462666829427
    ],
    "return_min": [
      0.0,
      83.71072308019035,
      208.12097190097407,
      239.92148547332886,
      268.6843518582533,
      280.06576497283703,
      282.01917265202314,
      295.1558484227942,
      298.3069802027448,
      300.2730247321967,
      303.5733979888646,
      300.2010388008713,
      303.3949109736265,
      311.38583689703665,
      308.21839791204366,
      310.94946877443516,
      310.7380146256782,
      314.98049407945916,
      316.6951429319435,
      312.89852900846034,
      317.0532202601144
    ],
    "return_max": [
      0.0,
      144.13088560307787,
      243.26151507819262,
      258.3523589277128,
      284.69111607794457,
      294.9564721690901,
      293.3938725953727,
      302.3812406071537,
      307.76009743397395,
      310.19942806728244,
      318.8399117442083,
      325.19780560017034,
      329.76142040658186,
      331.48452443108835,
      338.0915182663417,
      346.0040834716586,
      351.70003469072805,
      350.6643626913742,
      349.57505970477524,
      348.09314986523754,
      346.43931310577096
    ]
  },
  "h1hand_truck": {
    "time": [
      0.0,
      356.2344639081333,
      712.4689278162666,
      1068.7033917244,
      1424.9378556325332,
      1781.1723195406666,
      2137.4067834488,
      2493.6412473569335,
      2849.8757112650665,
      3206.1101751732,
      3562.3446390813333,
      3918.5791029894667,
      4274.8135668976,
      4631.048030805733,
      4987.282494713867,
      5343.516958622,
      5699.751422530133,
      6055.985886438267,
      6412.2203503464,
      6768.454814254534,
      7124.689278162667,
      7480.9237420708,
      7837.158205978933,
      8193.392669887067,
      8549.6271337952,
      8905.861597703333,
      9262.096061611466,
      9618.3305255196,
      9974.564989427734,
      10330.799453335867,
      10687.033917244,
      11043.268381152133,
      11399.502845060266,
      11755.7373089684,
      12111.971772876534,
      12468.206236784667,
      12824.4407006928,
      13180.675164600932,
      13536.909628509067,
      13893.1440924172,
      14249.378556325333,
      14605.613020233466,
      14961.8474841416,
      15318.081948049734,
      15674.316411957867,
      16030.550875866,
      16386.785339774135,
      16743.019803682266,
      17099.2542675904,
      17455.48873149853,
      17811.723195406666,
      18167.957659314798,
      18524.192123222932,
      18880.426587131067,
      19236.6610510392,
      19592.89551494733,
      19949.129978855468,
      20305.364442763603,
      20661.598906671734,
      21017.833370579865,
      21374.067834488
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0,
      13440000.0,
      14080000.0,
      14720000.0,
      15360000.0,
      16000000.0,
      16640000.0,
      17280000.0,
      17920000.0,
      18560000.0,
      19200000.0,
      19840000.0,
      20480000.0,
      21120000.0,
      21760000.0,
      22400000.0,
      23040000.0,
      23680000.0,
      24320000.0,
      24960000.0,
      25600000.0,
      26240000.0,
      26880000.0,
      27520000.0,
      28160000.0,
      28800000.0,
      29440000.0,
      30080000.0,
      30720000.0,
      31360000.0,
      32000000.0,
      32640000.0,
      33280000.0,
      33920000.0,
      34560000.0,
      35200000.0,
      35840000.0,
      36480000.0,
      37120000.0,
      37760000.0,
      38400000.0
    ],
    "return": [
      0.0,
      744.8859558105469,
      796.5100860595703,
      910.9717559814453,
      975.1925201416016,
      1047.324951171875,
      1099.5369873046875,
      1148.0682067871094,
      1201.0522766113281,
      1233.505615234375,
      1255.5220031738281,
      1290.6919860839844,
      1311.6923828125,
      1338.3526000976562,
      1359.3213195800781,
      1323.95654296875,
      1365.9458312988281,
      1372.3952026367188,
      1371.4238586425781,
      1380.5548706054688,
      1387.2607421875,
      1377.4356384277344,
      1405.5323791503906,
      1398.5776977539062,
      1386.4181823730469,
      1399.1890563964844,
      1414.0697021484375,
      1413.0535888671875,
      1426.5853576660156,
      1436.8981628417969,
      1447.1106567382812,
      1452.8192138671875,
      1458.5155639648438,
      1482.8174133300781,
      1488.4780883789062,
      1509.2735900878906,
      1529.7430114746094,
      1542.8193664550781,
      1523.0067443847656,
      1534.5915832519531,
      1547.8734130859375,
      1558.2222290039062,
      1559.4790344238281,
      1521.8999328613281,
      1579.9296264648438,
      1568.0743103027344,
      1592.0011596679688,
      1607.4393005371094,
      1601.7018432617188,
      1628.3120422363281,
      1632.6797180175781,
      1637.2363586425781,
      1640.4541015625,
      1648.7554626464844,
      1637.6644287109375,
      1660.0960693359375,
      1661.5935363769531,
      1654.3347778320312,
      1661.7125244140625,
      1665.4105834960938,
      1686.1488037109375
    ],
    "return_min": [
      0.0,
      728.8708482176052,
      762.0585259369802,
      843.073411732124,
      889.465099012336,
      998.3990660977275,
      1063.9706960921915,
      1116.6410384966878,
      1161.0117972409382,
      1195.7086321791687,
      1226.7667966679335,
      1246.3428404936096,
      1272.6613742372551,
      1296.5317436158239,
      1329.0087405178306,
      1273.7531020924337,
      1335.3961578228586,
      1342.6771680749928,
      1338.9942289987519,
      1354.398859758841,
      1352.2997701371219,
      1322.036434338705,
      1362.2402065457197,
      1348.2375712433736,
      1328.8611382153329,
      1353.1487389210438,
      1362.2909765642328,
      1378.9425298602578,
      1373.9676375490837,
      1387.940960280719,
      1394.5923024990384,
      1392.6599873879725,
      1397.4042979437356,
      1416.8406937577124,
      1436.3985979041825,
      1451.1102549614811,
      1462.6237026144347,
      1480.086465539093,
      1446.3469016733263,
      1463.1031736967145,
      1480.826246455255,
      1507.2520940243835,
      1484.954383969524,
      1427.5705673711657,
      1514.958677668042,
      1495.3212565792096,
      1524.5236672425701,
      1493.734800162037,
      1512.489867922995,
      1521.317076862614,
      1501.4009697149443,
      1512.9679254210876,
      1523.2808539998027,
      1516.759711486079,
      1521.9907509338339,
      1541.900269758106,
      1540.4093855281021,
      1540.2998088326399,
      1549.106670304241,
      1539.2902805167103,
      1557.2959035745841
    ],
    "return_max": [
      0.0,
      760.9010634034886,
      830.9616461821604,
      978.8701002307666,
      1060.9199412708672,
      1096.2508362460223,
      1135.1032785171835,
      1179.495375077531,
      1241.092755981718,
      1271.3025982895813,
      1284.2772096797228,
      1335.0411316743591,
      1350.7233913877449,
      1380.1734565794886,
      1389.6338986423257,
      1374.1599838450663,
      1396.4955047747976,
      1402.1132371984447,
      1403.8534882864044,
      1406.7108814520966,
      1422.2217142378781,
      1432.8348425167637,
      1448.8245517550615,
      1448.9178242644389,
      1443.975226530761,
      1445.229373871925,
      1465.8484277326422,
      1447.1646478741172,
      1479.2030777829475,
      1485.8553654028747,
      1499.629010977524,
      1512.9784403464025,
      1519.6268299859519,
      1548.7941329024438,
      1540.55757885363,
      1567.4369252143001,
      1596.862320334784,
      1605.5522673710632,
      1599.666587096205,
      1606.0799928071917,
      1614.92057971662,
      1609.192363983429,
      1634.0036848781322,
      1616.2292983514906,
      1644.9005752616456,
      1640.8273640262591,
      1659.4786520933674,
      1721.1438009121819,
      1690.9138186004425,
      1735.3070076100423,
      1763.958466320212,
      1761.5047918640687,
      1757.6273491251973,
      1780.7512138068898,
      1753.3381064880411,
      1778.291868913769,
      1782.7776872258041,
      1768.3697468314226,
      1774.318378523884,
      1791.5308864754772,
      1815.0017038472909
    ]
  },
  "h1hand_cube": {
    "time": [
      0.0,
      383.7138327153,
      767.4276654306,
      1151.1414981459,
      1534.8553308612,
      1918.5691635765002,
      2302.2829962918,
      2685.9968290071,
      3069.7106617224,
      3453.4244944377,
      3837.1383271530003,
      4220.8521598683,
      4604.5659925836,
      4988.2798252989,
      5371.9936580142,
      5755.7074907295,
      6139.4213234448,
      6523.1351561600995,
      6906.8489888754,
      7290.5628215907,
      7674.276654306001
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0
    ],
    "return": [
      0.0,
      13.080046653747559,
      17.67292849222819,
      21.029696782430012,
      22.918834050496418,
      24.50208346048991,
      26.14434051513672,
      35.192575454711914,
      47.09134292602539,
      60.91011047363281,
      78.56637700398763,
      158.25643920898438,
      161.40049743652344,
      135.81275049845377,
      156.24981689453125,
      225.83556111653647,
      166.00001017252603,
      213.50342814127603,
      196.95799255371094,
      205.33394368489584,
      221.19602966308594
    ],
    "return_min": [
      0.0,
      11.60227465184281,
      16.627264717234375,
      20.016001116689015,
      22.27803754584949,
      23.950662436438055,
      23.630521485969602,
      23.826143545159375,
      31.20715640142832,
      43.19868665817663,
      46.30510159872897,
      130.83772424309208,
      124.81023014354234,
      58.897895471155806,
      106.57978648931135,
      219.94269343957157,
      148.87724150076664,
      193.22438191225328,
      179.88170809603196,
      178.60399498737277,
      210.30042873135127
    ],
    "return_max": [
      0.0,
      14.557818655652307,
      18.718592267222007,
      22.04339244817101,
      23.559630555143347,
      25.053504484541765,
      28.658159544303835,
      46.55900736426445,
      62.97552945062246,
      78.621534289089,
      110.82765240924628,
      185.67515417487667,
      197.99076472950452,
      212.72760552575173,
      205.91984729975115,
      231.72842879350137,
      183.12277884428542,
      233.7824743702988,
      214.03427701138992,
      232.0638923824189,
      232.0916305948206
    ]
  },
  "h1hand_bookshelf_simple": {
    "time": [
      0.0,
      607.6057830167999,
      1215.2115660335999,
      1822.8173490504,
      2430.4231320671997,
      3038.028915084,
      3645.6346981008,
      4253.2404811176,
      4860.8462641343995,
      5468.4520471512,
      6076.057830168,
      6683.663613184801,
      7291.2693962016,
      7898.875179218399,
      8506.4809622352,
      9114.086745252,
      9721.692528268799,
      10329.298311285598,
      10936.9040943024,
      11544.5098773192,
      12152.115660336
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0
    ],
    "return": [
      0.0,
      72.59014129638672,
      632.5331013997396,
      482.05072021484375,
      417.5148289998372,
      487.9495900472005,
      716.89990234375,
      602.3393147786459,
      723.6810099283854,
      746.5069580078125,
      680.5458170572916,
      555.5288289388021,
      762.16845703125,
      761.0180053710938,
      791.8561401367188,
      807.6997273763021,
      812.0292765299479,
      816.4413655598959,
      816.0659993489584,
      819.2063598632812,
      821.7978922526041
    ],
    "return_min": [
      0.0,
      31.586202163025966,
      591.3560061650909,
      333.63710951300334,
      182.78757131870665,
      272.4163623170391,
      704.8062688039831,
      466.3347552851721,
      701.2404371259585,
      723.2593912345505,
      616.9144495010835,
      362.17470111103154,
      737.8159270729626,
      749.7294135850707,
      778.164806689675,
      801.360166533209,
      804.7945123252052,
      799.3652871282668,
      800.0447951141915,
      800.9554924523965,
      804.5781225227375
    ],
    "return_max": [
      0.0,
      113.59408042974746,
      673.7101966343884,
      630.4643309166842,
      652.2420866809678,
      703.4828177773619,
      728.9935358835169,
      738.3438742721196,
      746.1215827308123,
      769.7545247810745,
      744.1771846134998,
      748.8829567665728,
      786.5209869895374,
      772.3065971571168,
      805.5474735837626,
      814.0392882193952,
      819.2640407346905,
      833.5174439915249,
      832.0872035837252,
      837.457227274166,
      839.0176619824707
    ]
  },
  "h1hand_bookshelf_hard": {
    "time": [
      0.0,
      543.7358112938999,
      1087.4716225877999,
      1631.2074338817,
      2174.9432451755997,
      2718.6790564694998,
      3262.4148677634,
      3806.1506790573,
      4349.886490351199,
      4893.6223016450995,
      5437.3581129389995,
      5981.0939242329005,
      6524.8297355268,
      7068.5655468207,
      7612.3013581146,
      8156.0371694085,
      8699.772980702399,
      9243.5087919963,
      9787.244603290199,
      10330.9804145841,
      10874.716225877999,
      11418.452037171899,
      11962.187848465801,
      12505.9236597597,
      13049.6594710536,
      13593.395282347501,
      14137.1310936414,
      14680.8669049353,
      15224.6027162292,
      15768.3385275231,
      16312.074338817
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0,
      13440000.0,
      14080000.0,
      14720000.0,
      15360000.0,
      16000000.0,
      16640000.0,
      17280000.0,
      17920000.0,
      18560000.0,
      19200000.0
    ],
    "return": [
      0.0,
      115.04310989379883,
      414.3397623697917,
      392.4563496907552,
      384.3321024576823,
      445.73602294921875,
      444.00734456380206,
      507.5265604654948,
      502.8351338704427,
      563.632080078125,
      479.8868408203125,
      540.4158935546875,
      504.34425862630206,
      576.1414082845052,
      565.2438761393229,
      577.5482991536459,
      557.5013427734375,
      589.2844645182291,
      630.3101806640625,
      653.0324300130209,
      625.0805460611979,
      712.5912882486979,
      699.0077311197916,
      691.9513346354166,
      665.5681762695312,
      712.416259765625,
      694.9320068359375,
      723.7599080403646,
      704.8312174479166,
      718.2322794596354,
      723.0150553385416
    ],
    "return_min": [
      0.0,
      43.59245386852548,
      332.4218818003097,
      299.83306064986766,
      259.7826089424531,
      356.7439063459728,
      399.02623029313395,
      478.43961173592857,
      480.8452493513443,
      529.9966373726622,
      431.6378949090861,
      494.5170640027713,
      422.71565001450017,
      521.9929747241146,
      534.9803965407618,
      513.9383974775631,
      536.6718386033316,
      543.0112313037442,
      612.9851065450109,
      646.1570432133583,
      611.3883743554222,
      687.0628606877813,
      676.3860733803567,
      672.6699943456148,
      627.8278602083814,
      698.0179459658565,
      658.5159895805781,
      707.7512852510698,
      689.8015943037292,
      706.8565054687476,
      705.8978014554126
    ],
    "return_max": [
      0.0,
      186.4937659190722,
      496.2576429392737,
      485.0796387316427,
      508.8815959729115,
      534.7281395524647,
      488.9884588344702,
      536.613509195061,
      524.8250183895411,
      597.2675227835878,
      528.1357867315388,
      586.3147231066037,
      585.972867238104,
      630.2898418448959,
      595.5073557378839,
      641.1582008297287,
      578.3308469435434,
      635.557697732714,
      647.6352547831141,
      659.9078168126834,
      638.7727177669735,
      738.1197158096145,
      721.6293888592265,
      711.2326749252185,
      703.3084923306811,
      726.8145735653935,
      731.3480240912969,
      739.7685308296594,
      719.8608405921041,
      729.6080534505231,
      740.1323092216707
    ]
  },
  "h1hand_basketball": {
    "time": [
      0.0,
      302.93849988592,
      605.87699977184,
      908.8154996577599,
      1211.75399954368,
      1514.6924994295998,
      1817.6309993155198,
      2120.56949920144,
      2423.50799908736,
      2726.44649897328,
      3029.3849988591996,
      3332.32349874512,
      3635.2619986310397,
      3938.20049851696,
      4241.13899840288,
      4544.0774982888,
      4847.01599817472,
      5149.95449806064,
      5452.89299794656,
      5755.83149783248,
      6058.769997718399,
      6361.708497604321,
      6664.64699749024,
      6967.58549737616,
      7270.523997262079,
      7573.462497147999,
      7876.40099703392,
      8179.33949691984,
      8482.27799680576,
      8785.216496691679,
      9088.1549965776,
      9391.09349646352,
      9694.03199634944,
      9996.970496235359,
      10299.90899612128,
      10602.847496007198,
      10905.78599589312,
      11208.72449577904,
      11511.66299566496,
      11814.60149555088,
      12117.539995436799,
      12420.478495322719,
      12723.416995208641,
      13026.35549509456,
      13329.29399498048,
      13632.232494866399,
      13935.17099475232,
      14238.10949463824,
      14541.047994524159,
      14843.98649441008,
      15146.924994295998
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0,
      13440000.0,
      14080000.0,
      14720000.0,
      15360000.0,
      16000000.0,
      16640000.0,
      17280000.0,
      17920000.0,
      18560000.0,
      19200000.0,
      19840000.0,
      20480000.0,
      21120000.0,
      21760000.0,
      22400000.0,
      23040000.0,
      23680000.0,
      24320000.0,
      24960000.0,
      25600000.0,
      26240000.0,
      26880000.0,
      27520000.0,
      28160000.0,
      28800000.0,
      29440000.0,
      30080000.0,
      30720000.0,
      31360000.0,
      32000000.0
    ],
    "return": [
      0.0,
      15.71937624613444,
      20.09008280436198,
      28.070403416951496,
      37.13821029663086,
      48.140185038248696,
      80.68272908528645,
      124.40814971923828,
      126.71758778889973,
      161.22200520833334,
      202.98166147867838,
      204.3378473917643,
      293.4285481770833,
      315.3683675130208,
      264.8266194661458,
      278.62744140625,
      291.4298909505208,
      324.5673472086589,
      380.23256429036456,
      394.94849650065106,
      298.89170837402344,
      292.83021036783856,
      401.73631795247394,
      375.27061971028644,
      386.8580322265625,
      303.90362548828125,
      453.88189697265625,
      383.60216267903644,
      323.7223765055339,
      264.7662048339844,
      407.06358846028644,
      348.29319254557294,
      415.5738118489583,
      385.26619720458984,
      416.16399637858075,
      491.7182312011719,
      459.73570760091144,
      295.7273661295573,
      305.8920593261719,
      537.0791524251302,
      566.8707377115885,
      511.0398864746094,
      371.42539469401044,
      502.34424845377606,
      437.50067138671875,
      296.82629648844403,
      467.6920878092448,
      347.05420939127606,
      577.7560831705729,
      553.7951253255209,
      528.9782918294271
    ],
    "return_min": [
      0.0,
      14.283715681857537,
      16.631100081870674,
      24.850849036315818,
      25.682231879991985,
      40.36236009161064,
      62.133789054744106,
      84.60940729497763,
      81.26086178598888,
      90.6152240448114,
      104.67304686229286,
      94.84658618782778,
      198.80811217235924,
      260.1644753571341,
      154.88538218147525,
      223.0036764790103,
      261.8258052334307,
      207.3873303370363,
      286.2910705112556,
      381.83799263632784,
      245.92185692409578,
      247.45123796782846,
      364.1263718288159,
      353.57045366003666,
      364.68906452491575,
      209.5885532905373,
      390.04425163109977,
      314.08592505260197,
      213.68833766761594,
      214.40978111414398,
      329.92800186753425,
      229.6240291107526,
      337.0526110087683,
      199.91591366084674,
      268.50189022268603,
      431.10496585660894,
      434.0950085790193,
      121.5927500915191,
      259.7480611862015,
      473.60034355255095,
      467.59375605440545,
      416.2188627932545,
      249.31592931518838,
      411.0434362283788,
      261.3459217634894,
      176.60671178439534,
      300.33125904403,
      221.02134041025272,
      518.1882874852209,
      470.53052483418503,
      443.62534877420796
    ],
    "return_max": [
      0.0,
      17.155036810411342,
      23.549065526853287,
      31.289957797587174,
      48.59418871326973,
      55.91800998488675,
      99.2316691158288,
      164.20689214349892,
      172.1743137918106,
      231.8287863718553,
      301.29027609506386,
      313.82910859570086,
      388.0489841818074,
      370.5722596689075,
      374.76785675081635,
      334.2512063334897,
      321.03397666761094,
      441.74736408028144,
      474.1740580694735,
      408.0590003649743,
      351.8615598239511,
      338.2091827678487,
      439.346264076132,
      396.9707857605362,
      409.02699992820925,
      398.21869768602517,
      517.7195423142127,
      453.1184003054709,
      433.7564153434518,
      315.12262855382477,
      484.19917505303863,
      466.9623559803933,
      494.09501268914835,
      570.616480748333,
      563.8261025344755,
      552.3314965457348,
      485.3764066228036,
      469.86198216759556,
      352.03605746614227,
      600.5579612977095,
      666.1477193687715,
      605.8609101559642,
      493.5348600728325,
      593.6450606791733,
      613.6554210099481,
      417.04588119249274,
      635.0529165744597,
      473.0870783722994,
      637.3238788559248,
      637.0597258168567,
      614.3312348846463
    ]
  },
  "h1hand_window": {
    "time": [
      0.0,
      375.65864049644,
      751.31728099288,
      1126.97592148932,
      1502.63456198576,
      1878.2932024822,
      2253.95184297864,
      2629.61048347508,
      3005.26912397152,
      3380.9277644679596,
      3756.5864049644,
      4132.24504546084,
      4507.90368595728,
      4883.56232645372,
      5259.22096695016,
      5634.8796074466,
      6010.53824794304,
      6386.196888439479,
      6761.855528935919,
      7137.51416943236,
      7513.1728099288,
      7888.831450425239,
      8264.49009092168,
      8640.14873141812,
      9015.80737191456,
      9391.466012411,
      9767.12465290744,
      10142.78329340388,
      10518.44193390032,
      10894.10057439676,
      11269.7592148932,
      11645.41785538964,
      12021.07649588608,
      12396.73513638252,
      12772.393776878958,
      13148.052417375398,
      13523.711057871838,
      13899.369698368278,
      14275.02833886472,
      14650.68697936116,
      15026.3456198576,
      15402.00426035404,
      15777.662900850479,
      16153.321541346919,
      16528.98018184336,
      16904.6388223398,
      17280.29746283624,
      17655.95610333268,
      18031.61474382912,
      18407.27338432556,
      18782.932024822
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0,
      13440000.0,
      14080000.0,
      14720000.0,
      15360000.0,
      16000000.0,
      16640000.0,
      17280000.0,
      17920000.0,
      18560000.0,
      19200000.0,
      19840000.0,
      20480000.0,
      21120000.0,
      21760000.0,
      22400000.0,
      23040000.0,
      23680000.0,
      24320000.0,
      24960000.0,
      25600000.0,
      26240000.0,
      26880000.0,
      27520000.0,
      28160000.0,
      28800000.0,
      29440000.0,
      30080000.0,
      30720000.0,
      31360000.0,
      32000000.0
    ],
    "return": [
      0.0,
      9.344390233357748,
      52.42061551411947,
      165.07393646240234,
      269.02171834309894,
      336.8362223307292,
      303.15257771809894,
      354.14097086588544,
      428.8276774088542,
      233.65752283732095,
      354.2726542154948,
      446.46274820963544,
      386.8551534016927,
      488.44984944661456,
      527.6175537109375,
      499.61435953776044,
      520.1758015950521,
      533.7708740234375,
      517.2260945638021,
      615.2128092447916,
      586.9806111653646,
      631.5748697916666,
      625.1402994791666,
      522.4225260416666,
      661.9009399414062,
      621.3815714518229,
      593.1093546549479,
      609.5584716796875,
      645.6303304036459,
      606.4934692382812,
      629.7203369140625,
      619.4031575520834,
      616.0424601236979,
      619.2044474283854,
      614.5390421549479,
      615.4394938151041,
      560.7685038248698,
      474.59820556640625,
      470.9849739074707,
      668.3463948567709,
      682.6567586263021,
      464.4889322916667,
      636.9969889322916,
      627.0051981608073,
      670.6390177408854,
      633.2178141276041,
      626.3368123372396,
      667.2381591796875,
      550.3759155273438,
      638.7027587890625,
      613.3283081054688
    ],
    "return_min": [
      0.0,
      7.793400754588625,
      26.974647549456396,
      96.90356411186578,
      228.96287551323448,
      309.31272125827473,
      286.3705291802933,
      326.8671206873417,
      392.2532160264861,
      84.46841388823327,
      304.0752715628187,
      367.63128755485377,
      198.75912330174802,
      418.10049013236585,
      495.99406990234695,
      466.59337810178283,
      426.0956847450194,
      528.027321153031,
      427.74345783940487,
      588.4670108977814,
      572.1403582244072,
      621.8291590940654,
      580.3812827338901,
      411.5124914178696,
      624.5023420369643,
      602.0108924461694,
      561.3460675594077,
      584.0818549693116,
      585.6577994132044,
      524.2934694401896,
      556.6414078895589,
      597.6325510119733,
      578.4353616396595,
      535.9672354385903,
      548.0507414116869,
      510.63892367992145,
      495.25338547915845,
      273.7842564520836,
      166.61370163419144,
      594.1346175295835,
      654.6452652746367,
      289.90572041974156,
      555.2077996580139,
      529.1499181682586,
      639.152777939848,
      553.0639730276913,
      538.3629737339184,
      595.1634994947292,
      330.96572081605984,
      596.1039158315787,
      502.0886536727603
    ],
    "return_max": [
      0.0,
      10.895379712126871,
      77.86658347878254,
      233.24430881293893,
      309.0805611729634,
      364.35972340318364,
      319.93462625590456,
      381.4148210444292,
      465.4021387912223,
      382.84663178640864,
      404.4700368681709,
      525.2942088644171,
      574.9511835016374,
      558.7992087608633,
      559.241037519528,
      532.635340973738,
      614.2559184450848,
      539.514426893844,
      606.7087312881994,
      641.9586075918019,
      601.820864106322,
      641.3205804892679,
      669.8993162244432,
      633.3325606654637,
      699.2995378458482,
      640.7522504574764,
      624.872641750488,
      635.0350883900634,
      705.6028613940873,
      688.6934690363729,
      702.7992659385661,
      641.1737640921934,
      653.6495586077363,
      702.4416594181805,
      681.0273428982089,
      720.2400639502868,
      626.2836221705811,
      675.4121546807289,
      775.35624618075,
      742.5581721839583,
      710.6682519779675,
      639.0721441635918,
      718.7861782065694,
      724.8604781533559,
      702.1252575419228,
      713.371655227517,
      714.3106509405609,
      739.3128188646458,
      769.7861102386277,
      681.3016017465463,
      724.5679625381772
    ]
  },
  "h1hand_spoon": {
    "time": [
      0.0,
      322.2585738862,
      644.5171477724,
      966.7757216586,
      1289.0342955448,
      1611.292869431,
      1933.5514433172,
      2255.8100172034,
      2578.0685910896,
      2900.3271649758,
      3222.585738862
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0
    ],
    "return": [
      0.0,
      122.02554003397624,
      186.70377604166666,
      261.64577229817706,
      300.80299886067706,
      328.19361368815106,
      339.11742146809894,
      356.2308349609375,
      356.4991963704427,
      361.5694986979167,
      369.06699625651044
    ],
    "return_min": [
      0.0,
      34.19382261198194,
      91.68614757119323,
      220.42312853910647,
      278.48433900289217,
      320.38500523770017,
      325.4230361788222,
      331.20817004445007,
      354.7223789896127,
      349.114697321542,
      364.1910965162153
    ],
    "return_max": [
      0.0,
      209.85725745597054,
      281.7214045121401,
      302.86841605724766,
      323.12165871846196,
      336.00222213860195,
      352.8118067573757,
      381.25349987742493,
      358.27601375127267,
      374.0243000742914,
      373.9428959968056
    ]
  },
  "h1hand_package": {
    "time": [
      0.0,
      340.7822499229,
      681.5644998458,
      1022.3467497687001,
      1363.1289996916,
      1703.9112496145,
      2044.6934995374002,
      2385.4757494603,
      2726.2579993832,
      3067.0402493061,
      3407.822499229,
      3748.6047491519,
      4089.3869990748003,
      4430.1692489977,
      4770.9514989206,
      5111.7337488435005,
      5452.5159987664,
      5793.2982486893,
      6134.0804986122,
      6474.862748535101,
      6815.644998458
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0
    ],
    "return": [
      0.0,
      -8347.962727864584,
      -7953.91357421875,
      -7639.56884765625,
      -7962.0322265625,
      -8035.910807291667,
      -7031.6279296875,
      -7409.0107421875,
      -6930.870279947917,
      -7567.949869791667,
      -8071.77685546875,
      -6989.133951822917,
      -7464.599609375,
      -7308.195963541667,
      -7369.9765625,
      -7056.262532552083,
      -7344.883463541667,
      -6283.265950520833,
      -7419.428873697917,
      -7106.29931640625,
      -7487.24951171875
    ],
    "return_min": [
      0.0,
      -9291.683999288682,
      -8680.473541162266,
      -7798.53664407206,
      -8377.834608732313,
      -8635.369594101,
      -7305.690398586479,
      -7625.490297935072,
      -7398.502723648596,
      -7928.4496191424105,
      -8369.074711589812,
      -7316.171947275012,
      -8169.863535365709,
      -7698.752542894817,
      -7491.752086731574,
      -7174.744292881087,
      -7563.93196904456,
      -6491.269627424492,
      -7715.537928880829,
      -7283.65067627879,
      -7745.076350527821
    ],
    "return_max": [
      0.0,
      -7404.241456440486,
      -7227.353607275234,
      -7480.60105124044,
      -7546.229844392687,
      -7436.452020482336,
      -6757.565460788521,
      -7192.531186439928,
      -6463.237836247238,
      -7207.4501204409235,
      -7774.478999347688,
      -6662.095956370822,
      -6759.335683384291,
      -6917.6393841885165,
      -7248.201038268426,
      -6937.780772223079,
      -7125.834958038774,
      -6075.262273617174,
      -7123.319818515005,
      -6928.94795653371,
      -7229.422672909679
    ]
  },
  "h1hand_powerlift": {
    "time": [
      0.0,
      330.8597732899667,
      661.7195465799334,
      992.5793198699,
      1323.4390931598668,
      1654.2988664498334,
      1985.1586397398,
      2316.0184130297666,
      2646.8781863197337,
      2977.7379596097003,
      3308.597732899667,
      3639.4575061896335,
      3970.3172794796,
      4301.177052769567,
      4632.036826059533,
      4962.896599349499,
      5293.756372639467,
      5624.616145929434,
      5955.475919219401,
      6286.335692509367,
      6617.195465799334,
      6948.055239089301,
      7278.915012379267,
      7609.774785669233,
      7940.6345589592,
      8271.494332249167,
      8602.354105539134,
      8933.2138788291,
      9264.073652119067,
      9594.933425409034,
      9925.793198698999
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0,
      13440000.0,
      14080000.0,
      14720000.0,
      15360000.0,
      16000000.0,
      16640000.0,
      17280000.0,
      17920000.0,
      18560000.0,
      19200000.0
    ],
    "return": [
      0.0,
      113.51462300618489,
      121.9887212117513,
      114.68843332926433,
      136.49109395345053,
      133.07645670572916,
      150.48657735188803,
      147.2944081624349,
      160.43098958333334,
      160.48809814453125,
      171.36279296875,
      173.65789794921875,
      175.5028839111328,
      205.1602579752604,
      244.93697102864584,
      211.19544474283853,
      298.95766194661456,
      304.22186279296875,
      269.803716023763,
      286.91443888346356,
      311.9042460123698,
      286.8816223144531,
      310.73230997721356,
      300.2124938964844,
      296.96824137369794,
      301.94849650065106,
      325.45009358723956,
      312.3852844238281,
      329.8019307454427,
      309.90221150716144,
      324.70689900716144
    ],
    "return_min": [
      0.0,
      98.60086063853973,
      114.55728915382741,
      95.01551471352975,
      131.31232655463052,
      132.52894643766788,
      145.7905631000291,
      133.60270472550508,
      157.63405153985775,
      159.3252445285353,
      161.9094348876847,
      170.73693424256538,
      159.80489922254142,
      193.6200625634648,
      219.08587047631957,
      191.63435289037298,
      270.29798851974755,
      286.12178639622397,
      237.38621982654573,
      260.68960010790164,
      304.83943049232437,
      273.19842889543116,
      303.7634891685769,
      274.43171223168747,
      285.8977429185736,
      278.42592868276324,
      313.4876563980677,
      295.64470743599946,
      324.5820886809485,
      283.7021672876653,
      312.297612561625
    ],
    "return_max": [
      0.0,
      128.42838537383005,
      129.42015326967518,
      134.3613519449989,
      141.66986135227054,
      133.62396697379043,
      155.18259160374697,
      160.98611159936473,
      163.22792762680893,
      161.6509517605272,
      180.8161510498153,
      176.57886165587212,
      191.2008685997242,
      216.700453387056,
      270.7880715809721,
      230.75653659530408,
      327.6173353734816,
      322.32193918971353,
      302.2212122209803,
      313.1392776590255,
      318.96906153241525,
      300.5648157334751,
      317.70113078585024,
      325.9932755612813,
      308.03873982882226,
      325.4710643185389,
      337.4125307764114,
      329.1258614116568,
      335.02177280993686,
      336.1022557266576,
      337.1161854526979
    ]
  },
  "h1hand_room": {
    "time": [
      0.0,
      465.0374399698,
      930.0748799396,
      1395.1123199094,
      1860.1497598792,
      2325.187199849,
      2790.2246398188,
      3255.2620797886,
      3720.2995197584,
      4185.3369597282,
      4650.374399698,
      5115.4118396678,
      5580.4492796376,
      6045.4867196074,
      6510.5241595772,
      6975.561599547,
      7440.5990395168,
      7905.6364794866,
      8370.6739194564,
      8835.711359426201,
      9300.748799396
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0
    ],
    "return": [
      0.0,
      13.283664067586264,
      16.206466674804688,
      15.741510391235352,
      17.81355921427409,
      17.636091868082683,
      20.84869448343913,
      21.28005091349284,
      29.286340077718098,
      36.723029454549156,
      55.41912968953451,
      85.37582143147786,
      111.54248301188152,
      156.0373992919922,
      161.06229146321616,
      165.90640258789062,
      175.1955108642578,
      181.9734141031901,
      174.88094584147134,
      183.4038848876953,
      176.96692911783853
    ],
    "return_min": [
      0.0,
      10.70380901646449,
      15.550924139783536,
      13.495577650634877,
      15.432328173343322,
      15.783839514520572,
      17.184290172785886,
      20.759621257856587,
      14.243299280874174,
      26.23099362535826,
      21.424434200721606,
      46.457612262779534,
      76.395700351806,
      142.86692395305536,
      149.31479438123523,
      150.45741972383985,
      170.95027780126063,
      173.86462173894353,
      169.2223202556131,
      179.6609978535562,
      165.3778977118618
    ],
    "return_max": [
      0.0,
      15.863519118708037,
      16.86200920982584,
      17.98744313183583,
      20.194790255204857,
      19.488344221644795,
      24.51309879409237,
      21.800480569129093,
      44.32938087456202,
      47.21506528374005,
      89.41382517834741,
      124.29403060017619,
      146.68926567195703,
      169.207874630929,
      172.80978854519708,
      181.3553854519414,
      179.440743927255,
      190.08220646743666,
      180.53957142732958,
      187.14677192183441,
      188.55596052381526
    ]
  },
  "h1hand_insert_small": {
    "time": [
      0.0,
      315.23139559215,
      630.4627911843,
      945.6941867764501,
      1260.9255823686,
      1576.15697796075,
      1891.3883735529002,
      2206.6197691450498,
      2521.8511647372,
      2837.08256032935,
      3152.3139559215,
      3467.5453515136496,
      3782.7767471058005,
      4098.0081426979505,
      4413.2395382900995,
      4728.47093388225,
      5043.7023294744,
      5358.93372506655,
      5674.1651206587,
      5989.39651625085,
      6304.627911843
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0
    ],
    "return": [
      0.0,
      25.803948720296223,
      65.2731081644694,
      110.42922719319661,
      124.65948994954427,
      151.74050903320312,
      158.40826924641928,
      164.9043426513672,
      167.84961954752603,
      170.26196797688803,
      165.7256317138672,
      150.51225789388022,
      159.7950897216797,
      169.25322469075522,
      170.7041982014974,
      162.28902689615884,
      171.52276102701822,
      173.49874369303384,
      164.4512736002604,
      162.95905049641928,
      171.48112996419272
    ],
    "return_min": [
      0.0,
      16.259126578045933,
      27.20639444461152,
      103.35436633108236,
      114.25403470486306,
      144.04370182263935,
      153.86552855398466,
      161.79437375930542,
      166.41660146284676,
      168.28469510106487,
      161.68870399170868,
      121.0084021069551,
      141.81689245511308,
      166.0688410087526,
      167.85661523998058,
      155.22548705200677,
      168.48752737496002,
      171.0301362849067,
      156.70273979774515,
      149.2293200021275,
      169.0827938103662
    ],
    "return_max": [
      0.0,
      35.348770862546516,
      103.33982188432729,
      117.50408805531086,
      135.0649451942255,
      159.4373162437669,
      162.9510099388539,
      168.01431154342896,
      169.2826376322053,
      172.2392408527112,
      169.7625594360257,
      180.01611368080532,
      177.7732869882463,
      172.43760837275784,
      173.55178116301423,
      169.35256674031092,
      174.55799467907642,
      175.967351101161,
      172.19980740277566,
      176.68878099071105,
      173.87946611801922
    ]
  },
  "h1hand_insert_normal": {
    "time": [
      0.0,
      353.9304402593,
      707.8608805186,
      1061.7913207779,
      1415.7217610372,
      1769.6522012965002,
      2123.5826415558,
      2477.5130818151,
      2831.4435220744,
      3185.3739623336996,
      3539.3044025930003,
      3893.2348428523,
      4247.1652831116,
      4601.0957233709,
      4955.0261636302,
      5308.9566038895,
      5662.8870441488,
      6016.8174844080995,
      6370.747924667399,
      6724.6783649267,
      7078.608805186001
    ],
    "env_step": [
      0.0,
      640000.0,
      1280000.0,
      1920000.0,
      2560000.0,
      3200000.0,
      3840000.0,
      4480000.0,
      5120000.0,
      5760000.0,
      6400000.0,
      7040000.0,
      7680000.0,
      8320000.0,
      8960000.0,
      9600000.0,
      10240000.0,
      10880000.0,
      11520000.0,
      12160000.0,
      12800000.0
    ],
    "return": [
      0.0,
      24.10168711344401,
      54.070421854654946,
      99.81527709960938,
      135.47543080647787,
      130.91323852539062,
      152.1724065144857,
      162.59126790364584,
      164.9534657796224,
      168.67947896321616,
      168.1095987955729,
      168.2154744466146,
      174.81605529785156,
      176.54434204101562,
      179.82152811686197,
      179.19852193196616,
      182.65686543782553,
      184.3785146077474,
      187.4399668375651,
      185.9150187174479,
      191.98021443684897
    ],
    "return_min": [
      0.0,
      6.769507504418215,
      38.73171288128932,
      81.49871730558638,
      113.11107571013979,
      99.97948537079725,
      130.3948495447914,
      159.98263262262466,
      163.80269122995844,
      166.81903211046978,
      163.45338885046095,
      160.9069368380094,
      169.2923240593455,
      169.42940571808268,
      168.66802504056085,
      164.02913103172816,
      165.1513139904467,
      169.5900424483381,
      170.5853867351582,
      166.09199645001894,
      172.27076893820444
    ],
    "return_max": [
      0.0,
      41.43386672246981,
      69.40913082802058,
      118.13183689363237,
      157.83978590281595,
      161.846991679984,
      173.94996348417996,
      165.19990318466702,
      166.10424032928637,
      170.53992581596253,
      172.76580874068486,
      175.5240120552198,
      180.33978653635762,
      183.65927836394857,
      190.9750311931631,
      194.36791283220415,
      200.16241688520435,
      199.16698676715671,
      204.29454693997198,
      205.73804098487687,
      211.6896599354935
    ]
  }
}

================================================
FILE: data/isaaclab_result.json
================================================
{
  "Isaac-Velocity-Flat-G1-v0": {
    "FastTD3": {
      "time": [
        0.0,
        426.030303030303,
        852.060606060606,
        1278.090909090909,
        1704.121212121212,
        2130.151515151515,
        2556.181818181818,
        2982.212121212121,
        3408.242424242424,
        3834.2727272727275,
        4260.30303030303,
        4686.333333333333,
        5112.363636363636,
        5538.393939393939,
        5964.424242424242,
        6390.454545454545,
        6816.484848484848,
        7242.515151515152,
        7668.545454545455,
        8094.575757575758,
        8520.60606060606,
        8946.636363636364,
        9372.666666666666,
        9798.69696969697,
        10224.727272727272,
        10650.757575757576,
        11076.787878787878,
        11502.818181818182,
        11928.848484848484,
        12354.878787878788,
        12780.90909090909,
        13206.939393939394,
        13632.969696969696,
        14059.0
      ],
      "env_step": [
        0,
        20480000.0,
        40960000.0,
        61440000.0,
        81920000.0,
        102400000.0,
        122880000.0,
        143360000.0,
        163840000.0,
        184320000.0,
        204800000.0,
        225280000.0,
        245760000.0,
        266240000.0,
        286720000.0,
        307200000.0,
        327680000.0,
        348160000.0,
        368640000.0,
        389120000.0,
        409600000.0,
        430080000.0,
        450560000.0,
        471040000.0,
        491520000.0,
        512000000.0,
        532480000.0,
        552960000.0,
        573440000.0,
        593920000.0,
        614400000.0,
        634880000.0,
        655360000.0,
        675840000.0
      ],
      "return": [
        0.0,
        8.399159749348959,
        9.912333170572916,
        14.84921375910441,
        18.012757539749146,
        22.504486719767254,
        27.125051498413086,
        28.701950073242188,
        30.197755813598633,
        31.469010670979817,
        32.14909235636393,
        32.57625071207682,
        32.17204030354818,
        32.481005350748696,
        30.93257013956706,
        32.5035769144694,
        32.88990084330241,
        31.94870122273763,
        32.73742167154948,
        32.58588981628418,
        32.65592384338379,
        30.25904655456543,
        32.79962031046549,
        32.60353914896647,
        32.388885498046875,
        32.74258168538412,
        30.75256411234538,
        32.113457997639976,
        30.90188471476237,
        30.882394790649414,
        31.751652399698894,
        32.93280283610026,
        32.285691579182945,
        32.544677734375
      ],
      "return_min": [
        0.0,
        -0.4342708013469121,
        0.15131576165535776,
        3.056790615527323,
        4.690397268276504,
        13.184447519334512,
        21.25279365145474,
        22.70949840620638,
        25.157422436625925,
        27.96643827593808,
        29.69758316302085,
        30.56877892180152,
        29.76787859060772,
        30.696770112276443,
        28.644023779533903,
        30.465339016626782,
        30.92349475362472,
        29.35783947935273,
        30.895364911595788,
        31.14117205982312,
        31.353408955374725,
        27.429461255702815,
        31.54753402208512,
        31.30661763069416,
        31.116279961417973,
        31.529114984883087,
        29.048792317035463,
        30.636109312623667,
        29.16179985474695,
        29.983837058445577,
        29.91569292592529,
        32.48186031918378,
        31.36364585744209,
        32.12861466393829
      ],
      "return_max": [
        0.0,
        17.23259030004483,
        19.673350579490474,
        26.641636902681498,
        31.335117811221785,
        31.824525920199996,
        32.99730934537143,
        34.694401740277996,
        35.238089190571344,
        34.97158306602155,
        34.60060154970701,
        34.58372250235212,
        34.57620201648864,
        34.265240589220944,
        33.221116499600214,
        34.54181481231202,
        34.85630693298009,
        34.539562966122524,
        34.57947843150316,
        34.03060757274524,
        33.95843873139285,
        33.08863185342805,
        34.05170659884587,
        33.900460667238775,
        33.66149103467578,
        33.95604838588515,
        32.456335907655294,
        33.590806682656286,
        32.6419695747778,
        31.78095252285325,
        33.5876118734725,
        33.383745353016735,
        33.207737300923796,
        32.96074080481171
      ]
    },
    "null": {
      "time": [
        0.0,
        4.206,
        8.412,
        12.618,
        16.824,
        21.03,
        25.236,
        29.442,
        33.648,
        37.854,
        42.06,
        46.266,
        50.472,
        54.678,
        58.884,
        63.09,
        67.296,
        71.502,
        75.708,
        79.914,
        84.12,
        88.326,
        92.532,
        96.738,
        100.944,
        105.15,
        109.356,
        113.562,
        117.768,
        121.974,
        126.18,
        130.386,
        134.592,
        138.798,
        143.004,
        147.21,
        151.416,
        155.622,
        159.828,
        164.034,
        168.24,
        172.446,
        176.652,
        180.858,
        185.064,
        189.27,
        193.476,
        197.682,
        201.888,
        206.094,
        210.3,
        214.506,
        218.712,
        222.918,
        227.124,
        231.33,
        235.536,
        239.742,
        243.948,
        248.154,
        252.36,
        256.566,
        260.772,
        264.978,
        269.184,
        273.39,
        277.596,
        281.802,
        286.008,
        290.214,
        294.42,
        298.626,
        302.832,
        307.038,
        311.244,
        315.45,
        319.656,
        323.862,
        328.068,
        332.274,
        336.48,
        340.686,
        344.892,
        349.098,
        353.304,
        357.51,
        361.716,
        365.922,
        370.128,
        374.334,
        378.54,
        382.746,
        386.952,
        391.158,
        395.364,
        399.57,
        403.776,
        407.982,
        412.188,
        416.394,
        420.6,
        424.806,
        429.012,
        433.218,
        437.424,
        441.63,
        445.836,
        450.042,
        454.248,
        458.454,
        462.66,
        466.866,
        471.072,
        475.278,
        479.484,
        483.69,
        487.896,
        492.102,
        496.308,
        500.514,
        504.72,
        508.926,
        513.132,
        517.338,
        521.544,
        525.75,
        529.956,
        534.162,
        538.368,
        542.574,
        546.78,
        550.986,
        555.192,
        559.398,
        563.604,
        567.81,
        572.016,
        576.222,
        580.428,
        584.634,
        588.84,
        593.046,
        597.252,
        601.458,
        605.664,
        609.87,
        614.076,
        618.282,
        622.488,
        626.694,
        630.9,
        635.106,
        639.312,
        643.518,
        647.724,
        651.93,
        656.136,
        660.342,
        664.548,
        668.754,
        672.96,
        677.166,
        681.372,
        685.578,
        689.784,
        693.99,
        698.196,
        702.402,
        706.608,
        710.814,
        715.02,
        719.226,
        723.432,
        727.638,
        731.844,
        736.05,
        740.256,
        744.462,
        748.668,
        752.874,
        757.08,
        761.286,
        765.492,
        769.698,
        773.904,
        778.11,
        782.316,
        786.522,
        790.728,
        794.934,
        799.14,
        803.346,
        807.552,
        811.758,
        815.964,
        820.17,
        824.376,
        828.582,
        832.788,
        836.994,
        841.2,
        845.406,
        849.612,
        853.818,
        858.024,
        862.23,
        866.436,
        870.642,
        874.848,
        879.054,
        883.26,
        887.466,
        891.672,
        895.878,
        900.084,
        904.29,
        908.496,
        912.702,
        916.908,
        921.114,
        925.32,
        929.526,
        933.732,
        937.938,
        942.144,
        946.35,
        950.556,
        954.762,
        958.968,
        963.174,
        967.38,
        971.586,
        975.792,
        979.998,
        984.204,
        988.41,
        992.616,
        996.822,
        1001.028,
        1005.234,
        1009.44,
        1013.646,
        1017.852,
        1022.058,
        1026.264,
        1030.47,
        1034.676,
        1038.882,
        1043.088,
        1047.294,
        1051.5,
        1055.706,
        1059.912,
        1064.118,
        1068.324,
        1072.53,
        1076.736,
        1080.942,
        1085.148,
        1089.354,
        1093.56,
        1097.766,
        1101.972,
        1106.178,
        1110.384,
        1114.59,
        1118.796,
        1123.002,
        1127.208,
        1131.414,
        1135.62,
        1139.826,
        1144.032,
        1148.238,
        1152.444,
        1156.65,
        1160.856,
        1165.062,
        1169.268,
        1173.474,
        1177.68,
        1181.886,
        1186.092,
        1190.298,
        1194.504,
        1198.71,
        1202.916,
        1207.122,
        1211.328,
        1215.534,
        1219.74,
        1223.946,
        1228.152,
        1232.358,
        1236.564,
        1240.77,
        1244.976,
        1249.182,
        1253.388,
        1257.594,
        1261.8,
        1266.006,
        1270.212,
        1274.418,
        1278.624,
        1282.83,
        1287.036,
        1291.242,
        1295.448,
        1299.654,
        1303.86,
        1308.066,
        1312.272,
        1316.478,
        1320.684,
        1324.89,
        1329.096,
        1333.302,
        1337.508,
        1341.714,
        1345.92,
        1350.126,
        1354.332,
        1358.538,
        1362.744,
        1366.95,
        1371.156,
        1375.362,
        1379.568,
        1383.774,
        1387.98,
        1392.186,
        1396.392,
        1400.598,
        1404.804,
        1409.01,
        1413.216,
        1417.422,
        1421.628,
        1425.834,
        1430.04,
        1434.246,
        1438.452,
        1442.658,
        1446.864,
        1451.07,
        1455.276,
        1459.482,
        1463.688,
        1467.894,
        1472.1,
        1476.306,
        1480.512,
        1484.718,
        1488.924,
        1493.13,
        1497.336,
        1501.542,
        1505.748,
        1509.954,
        1514.16,
        1518.366,
        1522.572,
        1526.778,
        1530.984,
        1535.19,
        1539.396,
        1543.602,
        1547.808,
        1552.014,
        1556.22,
        1560.426,
        1564.632,
        1568.838,
        1573.044,
        1577.25,
        1581.456,
        1585.662,
        1589.868,
        1594.074,
        1598.28,
        1602.486,
        1606.692,
        1610.898,
        1615.104,
        1619.31,
        1623.516,
        1627.722,
        1631.928,
        1636.134,
        1640.34,
        1644.546,
        1648.752,
        1652.958,
        1657.164,
        1661.37,
        1665.576,
        1669.782,
        1673.988,
        1678.194,
        1682.4,
        1686.606,
        1690.812,
        1695.018,
        1699.224,
        1703.43,
        1707.636,
        1711.842,
        1716.048,
        1720.254,
        1724.46,
        1728.666,
        1732.872,
        1737.078,
        1741.284,
        1745.49,
        1749.696,
        1753.902,
        1758.108,
        1762.314,
        1766.52,
        1770.726,
        1774.932,
        1779.138,
        1783.344,
        1787.55,
        1791.756,
        1795.962,
        1800.168,
        1804.374,
        1808.58,
        1812.786,
        1816.992,
        1821.198,
        1825.404,
        1829.61,
        1833.816,
        1838.022,
        1842.228,
        1846.434,
        1850.64,
        1854.846,
        1859.052,
        1863.258,
        1867.464,
        1871.67,
        1875.876,
        1880.082,
        1884.288,
        1888.494,
        1892.7,
        1896.906,
        1901.112,
        1905.318,
        1909.524,
        1913.73,
        1917.936,
        1922.142,
        1926.348,
        1930.554,
        1934.76,
        1938.966,
        1943.172,
        1947.378,
        1951.584,
        1955.79,
        1959.996,
        1964.202,
        1968.408,
        1972.614,
        1976.82,
        1981.026,
        1985.232,
        1989.438,
        1993.644,
        1997.85,
        2002.056,
        2006.262,
        2010.468,
        2014.674,
        2018.88,
        2023.086,
        2027.292,
        2031.498,
        2035.704,
        2039.91,
        2044.116,
        2048.322,
        2052.528,
        2056.734,
        2060.94,
        2065.146,
        2069.352,
        2073.558,
        2077.764,
        2081.97,
        2086.176,
        2090.382,
        2094.588,
        2098.794,
        2103.0
      ],
      "env_step": [
        0,
        294912,
        393216,
        491520,
        589824,
        786432,
        884736,
        983040,
        1081344,
        1376256,
        1671168,
        1867776,
        1966080,
        2260992,
        2457600,
        2850816,
        3145728,
        3342336,
        3735552,
        3833856,
        4521984,
        5013504,
        5210112,
        5308416,
        5603328,
        5898240,
        6094848,
        6389760,
        6488064,
        6684672,
        7274496,
        7372800,
        7569408,
        7667712,
        7962624,
        8749056,
        9043968,
        9633792,
        9830400,
        9928704,
        10223616,
        10321920,
        10518528,
        10715136,
        10813440,
        11010048,
        11206656,
        11599872,
        11698176,
        12582912,
        12976128,
        13271040,
        13664256,
        13762560,
        13860864,
        14057472,
        14155776,
        14352384,
        14843904,
        15237120,
        15335424,
        15630336,
        15728640,
        16416768,
        16613376,
        17104896,
        17301504,
        17793024,
        17891328,
        17989632,
        18087936,
        18284544,
        19169280,
        19267584,
        19365888,
        19562496,
        19857408,
        19955712,
        20545536,
        20643840,
        20742144,
        20938752,
        21037056,
        21331968,
        21430272,
        21528576,
        21626880,
        22216704,
        22315008,
        22413312,
        23101440,
        23199744,
        23691264,
        24084480,
        24772608,
        24969216,
        25165824,
        25559040,
        25853952,
        25952256,
        26247168,
        26345472,
        26443776,
        26935296,
        27033600,
        27328512,
        27623424,
        27918336,
        28213248,
        28311552,
        28409856,
        28704768,
        28901376,
        29294592,
        30375936,
        30572544,
        30670848,
        31260672,
        31457280,
        31555584,
        31752192,
        32047104,
        32440320,
        32538624,
        32636928,
        32833536,
        33030144,
        33619968,
        33718272,
        33816576,
        33914880,
        35192832,
        35389440,
        35487744,
        35586048,
        36274176,
        36372480,
        36470784,
        36667392,
        37158912,
        37650432,
        38043648,
        38731776,
        38928384,
        39223296,
        39616512,
        39714816,
        40009728,
        40206336,
        40894464,
        41091072,
        41189376,
        41680896,
        41779200,
        41975808,
        42172416,
        42270720,
        42369024,
        42467328,
        42958848,
        43155456,
        43352064,
        43548672,
        43745280,
        43941888,
        45023232,
        45711360,
        46104576,
        46202880,
        46301184,
        46792704,
        47480832,
        47579136,
        47677440,
        47874048,
        48168960,
        48758784,
        49053696,
        49250304,
        49348608,
        49446912,
        49643520,
        49938432,
        50331648,
        50429952,
        50626560,
        51019776,
        52002816,
        52297728,
        52887552,
        53084160,
        53182464,
        53477376,
        53772288,
        53870592,
        54067200,
        54460416,
        54558720,
        55640064,
        55934976,
        56328192,
        56426496,
        56623104,
        57212928,
        58884096,
        59375616,
        59670528,
        60260352,
        60850176,
        60948480,
        61734912,
        61833216,
        62029824,
        62521344,
        62619648,
        63406080,
        63504384,
        63799296,
        64094208,
        64290816,
        64585728,
        65077248,
        65372160,
        65470464,
        65667072,
        65863680,
        66453504,
        66650112,
        66846720,
        67239936,
        67436544,
        68517888,
        68812800,
        69304320,
        69402624,
        69894144,
        70287360,
        70385664,
        70483968,
        70582272,
        70975488,
        71073792,
        71172096,
        71270400,
        71663616,
        71958528,
        72253440,
        72548352,
        72646656,
        72843264,
        73039872,
        73236480,
        73334784,
        73728000,
        74219520,
        74317824,
        74514432,
        74907648,
        75005952,
        75399168,
        75595776,
        75988992,
        76382208,
        76578816,
        76677120,
        77070336,
        77365248,
        77561856,
        77758464,
        78053376,
        78544896,
        78643200,
        78839808,
        78938112,
        79429632,
        79724544,
        79822848,
        79921152,
        80019456,
        80510976,
        80707584,
        81297408,
        81494016,
        81690624,
        82378752,
        82870272,
        84049920,
        84148224,
        84443136,
        85917696,
        86212608,
        86310912,
        86409216,
        86507520,
        86605824,
        86704128,
        86802432,
        86999040,
        87097344,
        87392256,
        87490560,
        87588864,
        87785472,
        88276992,
        88768512,
        88866816,
        89063424,
        89456640,
        89554944,
        89849856,
        89948160,
        90046464,
        90144768,
        90439680,
        90931200,
        91029504,
        91127808,
        91619328,
        91815936,
        92110848,
        92209152,
        92504064,
        92798976,
        93388800,
        93487104,
        93683712,
        93880320,
        93978624,
        94470144,
        95256576,
        95354880,
        95649792,
        95748096,
        95846400,
        96141312,
        96239616,
        96337920,
        96829440,
        96927744,
        97320960,
        97517568,
        97615872,
        97714176,
        98304000,
        98697216,
        99090432,
        99287040,
        99385344,
        99778560,
        99876864,
        100073472,
        100171776,
        100368384,
        100564992,
        100958208,
        101154816,
        101253120,
        101744640,
        101842944,
        102137856,
        102334464,
        102432768,
        102825984,
        102924288,
        103120896,
        103219200,
        103710720,
        103809024,
        103907328,
        104103936,
        104398848,
        105185280,
        105381888,
        105480192,
        105971712,
        106070016,
        106659840,
        106758144,
        106856448,
        106954752,
        107053056,
        107249664,
        107347968,
        107446272,
        107839488,
        108232704,
        108429312,
        108822528,
        109314048,
        109707264,
        109805568,
        110100480,
        110198784,
        110788608,
        110985216,
        111083520,
        111280128,
        111378432,
        111575040,
        112164864,
        112656384,
        112754688,
        112852992,
        113541120,
        114229248,
        114327552,
        114524160,
        114622464,
        114917376,
        115015680,
        115212288,
        115900416,
        116195328,
        116490240,
        117080064,
        117178368,
        117374976,
        117768192,
        118456320,
        119734272,
        120127488,
        120324096,
        120422400,
        121012224,
        121405440,
        121602048,
        121700352,
        122191872,
        122388480,
        123076608,
        123568128,
        123666432,
        124256256,
        124354560,
        124649472,
        124747776,
        124944384,
        125239296,
        125337600,
        125632512,
        125927424,
        126418944,
        126517248,
        126713856,
        127401984,
        128188416,
        128286720,
        128581632,
        128876544,
        129564672,
        130056192,
        131137536,
        131334144,
        131530752,
        132120576,
        132218880,
        132513792,
        133005312,
        133890048,
        134086656,
        134184960,
        134774784,
        135364608,
        135561216,
        136249344,
        136347648,
        136544256,
        136740864,
        137527296,
        137625600,
        138215424,
        138412032,
        138608640,
        139001856,
        139100160,
        139198464,
        139296768,
        139395072,
        139493376,
        139984896,
        140181504,
        140279808,
        141066240,
        141656064,
        141852672,
        142049280,
        142540800,
        142737408,
        143032320,
        143228928,
        143818752,
        143917056,
        144113664,
        144211968,
        144900096,
        145391616,
        146178048,
        146276352,
        146374656,
        146767872,
        146964480
      ],
      "return": [
        0.0,
        -4.244887483882465,
        -6.062188769976298,
        -5.946233414014181,
        -5.934175154368082,
        -5.931420200665792,
        -5.926097372372946,
        -5.890506885846456,
        -5.845593395233155,
        -5.745785452524821,
        -5.6373031155268345,
        -5.537198613484701,
        -5.437902429898579,
        -5.354564905166626,
        -5.315855964024862,
        -5.305097824732463,
        -5.233588047027588,
        -5.21569048722585,
        -5.142245790163676,
        -5.169298741022746,
        -5.144494528770447,
        -5.092627458572388,
        -5.067925599416097,
        -5.0944153579076135,
        -5.075692962010701,
        -5.080377626419067,
        -5.094903222719828,
        -5.125536025365193,
        -5.115845588048299,
        -5.232694799105326,
        -5.326773929595947,
        -5.540815292994181,
        -5.565265787442525,
        -5.767993720372519,
        -6.0354754002889,
        -6.422926289240519,
        -6.6024469566345205,
        -7.196066060066222,
        -7.374582826296488,
        -7.664462378819784,
        -8.25899796485901,
        -8.364871314366658,
        -8.789411764144896,
        -8.797979049682617,
        -9.422961320877077,
        -9.468292330106099,
        -9.801179609298705,
        -9.881031123797099,
        -10.184216540654502,
        -10.079860796928406,
        -9.81778653462728,
        -9.665794699986776,
        -9.04004868030548,
        -7.917503507932028,
        -6.944561026891073,
        -6.476619219779969,
        -6.198991281986236,
        -5.729001940091451,
        -4.624928730726242,
        -3.9058311102787653,
        -3.415297171473503,
        -3.0760760113100214,
        -2.5505499145636956,
        -1.384848330305734,
        -0.2722055218198026,
        0.37754478539495423,
        0.9910718053362021,
        1.9497537283304458,
        2.0353074125107375,
        2.7150473963220914,
        3.57238434004287,
        4.393549767136574,
        5.22253050327301,
        5.594326151609422,
        5.980829033851624,
        6.248111243247986,
        6.301437652111052,
        6.819287878672282,
        7.668490665753683,
        7.9056279023488365,
        8.528459188143414,
        8.9157959318161,
        9.004425404866536,
        9.240382947921754,
        10.005460894107818,
        10.388603275616964,
        10.506964916388194,
        11.041883727709452,
        11.13649615128835,
        11.725313129425048,
        12.456134204069775,
        12.757048015594483,
        13.031923011144002,
        13.684242639541628,
        14.265029729207356,
        14.391678101221721,
        14.820610736211142,
        15.028353486855826,
        15.586650098164876,
        16.098702065149944,
        16.480080706278486,
        16.47846872806549,
        16.60223277727763,
        16.996064742406208,
        16.962005354563395,
        17.414829864501954,
        17.879654438495635,
        18.075921009381613,
        18.59587174097697,
        18.768117705980938,
        19.066952754656473,
        19.140114987691245,
        19.385992137591046,
        19.82112768173218,
        20.07494811375936,
        20.38481765429179,
        20.64120300292969,
        20.944377085367837,
        21.046505676905316,
        21.219120070139567,
        21.259444128672282,
        21.437351309458418,
        21.490676719347636,
        21.614972871144612,
        21.73858602523804,
        21.836027539571123,
        21.938353093465167,
        22.12277064641317,
        22.096696147918703,
        22.379330412546793,
        22.3636573155721,
        22.667728048960367,
        22.804457734425863,
        22.989772860209147,
        22.81014023224513,
        22.752846819559732,
        22.87626227140427,
        23.268132722377782,
        23.450477498372397,
        23.49831963221232,
        23.642868830362957,
        23.749657942454018,
        23.829254946708676,
        23.89071097215017,
        23.95269861221313,
        23.80049831072489,
        23.97088320493698,
        24.035058199564617,
        23.987095406850177,
        24.143753560384113,
        24.161667313575744,
        24.20282023747762,
        24.515907077789308,
        24.48859402338664,
        24.65326049168905,
        24.695785293579103,
        24.68093146642049,
        24.64084663550059,
        24.774629052480062,
        24.721785335540773,
        24.80927684466044,
        24.64293538570404,
        24.611320269902546,
        24.890653877258302,
        25.03125279744466,
        25.04939617792765,
        25.056911741892495,
        25.012861070632937,
        25.006108971436817,
        25.19971148808797,
        25.244278527895606,
        25.243163836797077,
        25.20921319325765,
        25.25394191265106,
        25.238686189651492,
        25.24318765004476,
        25.405525760650633,
        25.469966049194337,
        25.493687133789063,
        25.4357274723053,
        25.59193923314412,
        25.664110005696614,
        25.589963239034017,
        25.78192579905192,
        25.539709254900615,
        25.59874127229055,
        25.71211055119832,
        25.83485471089681,
        25.849721884727483,
        25.78178211212158,
        25.905887908935544,
        26.03030003229777,
        25.978182222048442,
        26.085983327229815,
        26.114759820302325,
        26.203529485066735,
        26.11474272966385,
        26.242303619384767,
        26.12391219774882,
        25.88732498884201,
        26.3737934366862,
        26.40230457305908,
        26.32623062769572,
        26.447194538116452,
        26.39948532104492,
        26.48303238550822,
        26.58627212524414,
        26.62195227940877,
        26.305497008959453,
        26.5571306848526,
        26.676981925964355,
        26.715099360148113,
        26.7491853205363,
        26.77513189315796,
        26.77031274795532,
        26.699470434188843,
        26.697296330134076,
        26.761435826619465,
        26.773332411448163,
        26.817168051401776,
        26.812312013308212,
        26.730486590067546,
        26.980856755574546,
        26.768192443052925,
        26.75973533630371,
        26.726683316230776,
        26.629371005694072,
        26.737455078760785,
        27.04755596160889,
        26.996712977091473,
        27.15843843460083,
        27.056537364323933,
        27.07920110066732,
        27.063170181910195,
        26.999029286702477,
        26.95050971984863,
        27.028388187090556,
        27.162973842620847,
        27.03351854960124,
        26.927428197860717,
        27.040847080548605,
        27.197179768880208,
        27.07626317977905,
        27.213608862559,
        27.158017387390135,
        27.26690890630086,
        27.162100307146705,
        27.21130386193593,
        27.205319883028665,
        27.24621044794718,
        27.22654726187388,
        27.380611095428467,
        27.297355159123736,
        27.317657216389975,
        27.35010398864746,
        27.37887262980143,
        27.204053241411845,
        27.41668393452962,
        27.38345681508382,
        27.39409785588582,
        27.458933315277097,
        27.5122291692098,
        27.437938969930016,
        27.426795717875166,
        27.50698534965515,
        27.471194496154784,
        27.481666673024495,
        27.474861726760864,
        27.534579620361328,
        27.485984223683676,
        27.63851408004761,
        27.601514314015706,
        27.468947197596233,
        27.463055524826046,
        27.63535165786743,
        27.540319763819372,
        27.61072525024414,
        27.618154637018836,
        27.742102012634277,
        27.669274317423504,
        27.588603134155274,
        27.560866799354557,
        27.558601020177207,
        27.617678836186727,
        27.334658497174576,
        27.753905919392903,
        27.710659249623617,
        27.830444075266524,
        27.792324670155846,
        27.715141557057695,
        27.80116782506307,
        27.822862389882403,
        27.925147520701092,
        27.828692003885905,
        27.774182446797692,
        27.82865638097127,
        27.87010919570923,
        27.88524030049642,
        27.780755894978842,
        27.869576110839844,
        27.8847115389506,
        27.88112828572591,
        27.95226758956909,
        27.9632572110494,
        27.98185961405436,
        27.900285018285114,
        27.749773004849754,
        27.95440636952718,
        27.897387890815736,
        27.774074436823526,
        27.851776852607728,
        27.88595195611318,
        27.957040735880536,
        28.01861265818278,
        28.072463569641116,
        27.927986801465355,
        28.054341169993084,
        28.04882469813029,
        28.08775482813517,
        28.013588581085205,
        27.971034762064615,
        27.939692214330037,
        27.961951595942182,
        27.91120104789734,
        27.912012211481727,
        27.946552640597023,
        28.13232330322266,
        28.155323626200357,
        28.21534303029378,
        28.105512018203736,
        28.12738781611124,
        28.18438720703125,
        28.129107367197673,
        28.202164802551266,
        28.183705285390218,
        28.11675905227661,
        28.143270753224694,
        28.115149936676023,
        27.962960600852966,
        28.082834936777747,
        28.17713045756022,
        28.071439940134685,
        28.036277154286704,
        28.015016555786133,
        27.74030689318975,
        28.143797952334086,
        28.083258043924967,
        28.143300120035807,
        28.22578639984131,
        28.189498087565102,
        28.28595506032308,
        28.153611634572346,
        28.271268889109294,
        28.1683358446757,
        28.215656159718833,
        28.146134676933286,
        28.001276264190675,
        27.983188803990686,
        27.911292719841004,
        28.1467857392629,
        28.130204054514564,
        28.11707345326742,
        28.18377244313558,
        28.046776949564617,
        28.17007304509481,
        28.181098283131917,
        28.141192302703857,
        28.183331216176352,
        28.045621960957845,
        28.206850026448564,
        28.094409753481546,
        28.100912955602013,
        28.095957455635073,
        28.08301568508148,
        27.989888830184935,
        27.994859463373817,
        28.122985763549806,
        28.221680157979325,
        28.20320109049479,
        28.07082467556,
        28.08045124053955,
        28.234146792093913,
        28.24658082962036,
        28.167981278101603,
        28.19567700703939,
        28.181134967803956,
        28.086012459596002,
        28.175039056142168,
        28.172370891571045,
        28.007196400960286,
        28.25803017298381,
        28.184969975153606,
        28.25297903696696,
        28.171354506810506,
        28.238894797960914,
        28.285828113555908,
        27.95717629591624,
        28.18456648508708,
        28.211361389160157,
        28.155993226369223,
        28.133868006070454,
        28.174914951324464,
        28.19500195185343,
        28.166361468633017,
        27.803325554529824,
        27.919510776996614,
        28.102442877292635,
        28.161700401306152,
        28.24873664855957,
        28.270584068298344,
        28.14114588101705,
        28.200696325302122,
        28.242460556030277,
        28.08400403817495,
        28.200781688690185,
        28.170070962905882,
        28.197558924357097,
        28.121
Download .txt
gitextract_7_u6euyd/

├── .gitignore
├── .pre-commit-config.yaml
├── LICENSE
├── README.md
├── data/
│   ├── humanoidbench_result.json
│   ├── isaaclab_result.json
│   └── playground_result.json
├── fast_td3/
│   ├── __init__.py
│   ├── environments/
│   │   ├── humanoid_bench_env.py
│   │   ├── isaaclab_env.py
│   │   ├── mtbench_env.py
│   │   └── mujoco_playground_env.py
│   ├── fast_td3.py
│   ├── fast_td3_deploy.py
│   ├── fast_td3_simbav2.py
│   ├── fast_td3_utils.py
│   ├── hyperparams.py
│   ├── train.py
│   ├── train_multigpu.py
│   └── training_notebook.ipynb
├── requirements/
│   ├── requirements.txt
│   ├── requirements_isaacgym.txt
│   └── requirements_playground.txt
├── setup.py
└── sim2real.md
Download .txt
SYMBOL INDEX (162 symbols across 11 files)

FILE: fast_td3/environments/humanoid_bench_env.py
  function make_env (line 17) | def make_env(env_name, rank, render_mode=None, seed=0):
  class HumanoidBenchEnv (line 51) | class HumanoidBenchEnv:
    method __init__ (line 54) | def __init__(self, env_name, num_envs=1, render_mode=None, device=None):
    method reset (line 84) | def reset(self):
    method render (line 92) | def render(self):
    method step (line 98) | def step(self, actions):

FILE: fast_td3/environments/isaaclab_env.py
  class IsaacLabEnv (line 7) | class IsaacLabEnv:
    method __init__ (line 10) | def __init__(
    method reset (line 48) | def reset(self, random_start_init: bool = True) -> torch.Tensor:
    method reset_with_critic_obs (line 57) | def reset_with_critic_obs(self) -> tuple[torch.Tensor, torch.Tensor]:
    method step (line 61) | def step(
    method render (line 80) | def render(self):

FILE: fast_td3/environments/mtbench_env.py
  class MTBenchEnv (line 10) | class MTBenchEnv:
    method __init__ (line 11) | def __init__(
    method reset (line 62) | def reset(self) -> torch.Tensor:
    method step (line 72) | def step(
    method render (line 90) | def render(self):

FILE: fast_td3/environments/mujoco_playground_env.py
  class PlaygroundEvalEnvWrapper (line 8) | class PlaygroundEvalEnvWrapper:
    method __init__ (line 9) | def __init__(
    method reset (line 43) | def reset(self):
    method step (line 51) | def step(self, actions):
    method render_trajectory (line 61) | def render_trajectory(self, trajectory):
  function make_env (line 77) | def make_env(

FILE: fast_td3/fast_td3.py
  class DistributionalQNetwork (line 6) | class DistributionalQNetwork(nn.Module):
    method __init__ (line 7) | def __init__(
    method forward (line 31) | def forward(self, obs: torch.Tensor, actions: torch.Tensor) -> torch.T...
    method projection (line 36) | def projection(
  class Critic (line 84) | class Critic(nn.Module):
    method __init__ (line 85) | def __init__(
    method forward (line 120) | def forward(self, obs: torch.Tensor, actions: torch.Tensor) -> torch.T...
    method projection (line 123) | def projection(
    method get_value (line 152) | def get_value(self, probs: torch.Tensor) -> torch.Tensor:
  class Actor (line 157) | class Actor(nn.Module):
    method __init__ (line 158) | def __init__(
    method forward (line 196) | def forward(self, obs: torch.Tensor) -> torch.Tensor:
    method explore (line 202) | def explore(
  class MultiTaskActor (line 228) | class MultiTaskActor(Actor):
    method __init__ (line 229) | def __init__(self, num_tasks: int, task_embedding_dim: int, *args, **k...
    method forward (line 237) | def forward(self, obs: torch.Tensor) -> torch.Tensor:
  class MultiTaskCritic (line 247) | class MultiTaskCritic(Critic):
    method __init__ (line 248) | def __init__(self, num_tasks: int, task_embedding_dim: int, *args, **k...
    method forward (line 256) | def forward(self, obs: torch.Tensor, actions: torch.Tensor) -> torch.T...
    method projection (line 265) | def projection(

FILE: fast_td3/fast_td3_deploy.py
  class Policy (line 10) | class Policy(nn.Module):
    method __init__ (line 11) | def __init__(
    method forward (line 65) | def forward(self, obs: torch.Tensor) -> torch.Tensor:
    method act (line 71) | def act(self, obs: torch.Tensor) -> torch.distributions.Normal:
  function load_policy (line 76) | def load_policy(checkpoint_path):

FILE: fast_td3/fast_td3_simbav2.py
  function l2normalize (line 7) | def l2normalize(
  class Scaler (line 14) | class Scaler(nn.Module):
    method __init__ (line 19) | def __init__(
    method forward (line 30) | def forward(self, x: torch.Tensor) -> torch.Tensor:
  class HyperDense (line 34) | class HyperDense(nn.Module):
    method __init__ (line 39) | def __init__(self, in_dim: int, hidden_dim: int, device: torch.device ...
    method forward (line 44) | def forward(self, x: torch.Tensor) -> torch.Tensor:
  class HyperMLP (line 48) | class HyperMLP(nn.Module):
    method __init__ (line 53) | def __init__(
    method forward (line 69) | def forward(self, x: torch.Tensor) -> torch.Tensor:
  class HyperEmbedder (line 79) | class HyperEmbedder(nn.Module):
    method __init__ (line 84) | def __init__(
    method forward (line 99) | def forward(self, x: torch.Tensor) -> torch.Tensor:
  class HyperLERPBlock (line 111) | class HyperLERPBlock(nn.Module):
    method __init__ (line 116) | def __init__(
    method forward (line 142) | def forward(self, x: torch.Tensor) -> torch.Tensor:
  class HyperTanhPolicy (line 152) | class HyperTanhPolicy(nn.Module):
    method __init__ (line 157) | def __init__(
    method forward (line 171) | def forward(self, x: torch.Tensor) -> torch.Tensor:
  class HyperCategoricalValue (line 180) | class HyperCategoricalValue(nn.Module):
    method __init__ (line 185) | def __init__(
    method forward (line 199) | def forward(self, x: torch.Tensor) -> torch.Tensor:
  class DistributionalQNetwork (line 206) | class DistributionalQNetwork(nn.Module):
    method __init__ (line 207) | def __init__(
    method forward (line 261) | def forward(self, obs: torch.Tensor, actions: torch.Tensor) -> torch.T...
    method projection (line 268) | def projection(
  class Critic (line 316) | class Critic(nn.Module):
    method __init__ (line 317) | def __init__(
    method forward (line 373) | def forward(self, obs: torch.Tensor, actions: torch.Tensor) -> torch.T...
    method projection (line 376) | def projection(
    method get_value (line 405) | def get_value(self, probs: torch.Tensor) -> torch.Tensor:
  class Actor (line 410) | class Actor(nn.Module):
    method __init__ (line 411) | def __init__(
    method forward (line 471) | def forward(self, obs: torch.Tensor) -> torch.Tensor:
    method explore (line 478) | def explore(
  class MultiTaskActor (line 504) | class MultiTaskActor(Actor):
    method __init__ (line 505) | def __init__(self, num_tasks: int, task_embedding_dim: int, *args, **k...
    method forward (line 513) | def forward(self, obs: torch.Tensor) -> torch.Tensor:
  class MultiTaskCritic (line 523) | class MultiTaskCritic(Critic):
    method __init__ (line 524) | def __init__(self, num_tasks: int, task_embedding_dim: int, *args, **k...
    method forward (line 532) | def forward(self, obs: torch.Tensor, actions: torch.Tensor) -> torch.T...
    method projection (line 541) | def projection(

FILE: fast_td3/fast_td3_utils.py
  class SimpleReplayBuffer (line 12) | class SimpleReplayBuffer(nn.Module):
    method __init__ (line 13) | def __init__(
    method extend (line 90) | def extend(
    method sample (line 125) | def sample(self, batch_size: int):
  class EmpiricalNormalization (line 402) | class EmpiricalNormalization(nn.Module):
    method __init__ (line 405) | def __init__(self, shape, device, eps=1e-2, until=None):
    method mean (line 424) | def mean(self):
    method std (line 428) | def std(self):
    method forward (line 432) | def forward(
    method update (line 448) | def update(self, x):
    method inverse (line 496) | def inverse(self, y):
  class RewardNormalizer (line 500) | class RewardNormalizer(nn.Module):
    method __init__ (line 501) | def __init__(
    method _scale_reward (line 518) | def _scale_reward(self, rewards: torch.Tensor) -> torch.Tensor:
    method update_stats (line 525) | def update_stats(
    method forward (line 540) | def forward(self, rewards: torch.Tensor) -> torch.Tensor:
  class PerTaskEmpiricalNormalization (line 544) | class PerTaskEmpiricalNormalization(nn.Module):
    method __init__ (line 547) | def __init__(
    method forward (line 582) | def forward(
    method update (line 615) | def update(self, x: torch.Tensor, task_ids: torch.Tensor):
  class PerTaskRewardNormalizer (line 656) | class PerTaskRewardNormalizer(nn.Module):
    method __init__ (line 657) | def __init__(
    method _scale_reward (line 684) | def _scale_reward(
    method update_stats (line 706) | def update_stats(
    method forward (line 739) | def forward(self, rewards: torch.Tensor, task_ids: torch.Tensor) -> to...
  function cpu_state (line 750) | def cpu_state(sd):
  function save_params (line 755) | def save_params(
  function get_ddp_state_dict (line 795) | def get_ddp_state_dict(model):
  function load_ddp_state_dict (line 802) | def load_ddp_state_dict(model, state_dict):
  function mark_step (line 811) | def mark_step():

FILE: fast_td3/hyperparams.py
  class BaseArgs (line 7) | class BaseArgs:
  function get_args (line 129) | def get_args():
  class HumanoidBenchArgs (line 205) | class HumanoidBenchArgs(BaseArgs):
  class H1HandReachArgs (line 211) | class H1HandReachArgs(HumanoidBenchArgs):
  class H1HandBalanceSimpleArgs (line 218) | class H1HandBalanceSimpleArgs(HumanoidBenchArgs):
  class H1HandBalanceHardArgs (line 224) | class H1HandBalanceHardArgs(HumanoidBenchArgs):
  class H1HandPoleArgs (line 230) | class H1HandPoleArgs(HumanoidBenchArgs):
  class H1HandTruckArgs (line 236) | class H1HandTruckArgs(HumanoidBenchArgs):
  class H1HandMazeArgs (line 242) | class H1HandMazeArgs(HumanoidBenchArgs):
  class H1HandPushArgs (line 249) | class H1HandPushArgs(HumanoidBenchArgs):
  class H1HandBasketballArgs (line 257) | class H1HandBasketballArgs(HumanoidBenchArgs):
  class H1HandWindowArgs (line 265) | class H1HandWindowArgs(HumanoidBenchArgs):
  class H1HandPackageArgs (line 271) | class H1HandPackageArgs(HumanoidBenchArgs):
  class H1HandTruckArgs (line 278) | class H1HandTruckArgs(HumanoidBenchArgs):
  class MuJoCoPlaygroundArgs (line 285) | class MuJoCoPlaygroundArgs(BaseArgs):
  class MTBenchArgs (line 296) | class MTBenchArgs(BaseArgs):
  class MetaWorldMT10Args (line 310) | class MetaWorldMT10Args(MTBenchArgs):
  class MetaWorldMT50Args (line 321) | class MetaWorldMT50Args(MTBenchArgs):
  class G1JoystickFlatTerrainArgs (line 333) | class G1JoystickFlatTerrainArgs(MuJoCoPlaygroundArgs):
  class G1JoystickRoughTerrainArgs (line 339) | class G1JoystickRoughTerrainArgs(MuJoCoPlaygroundArgs):
  class T1JoystickFlatTerrainArgs (line 345) | class T1JoystickFlatTerrainArgs(MuJoCoPlaygroundArgs):
  class T1JoystickRoughTerrainArgs (line 351) | class T1JoystickRoughTerrainArgs(MuJoCoPlaygroundArgs):
  class T1LowDofJoystickFlatTerrainArgs (line 357) | class T1LowDofJoystickFlatTerrainArgs(MuJoCoPlaygroundArgs):
  class T1LowDofJoystickRoughTerrainArgs (line 363) | class T1LowDofJoystickRoughTerrainArgs(MuJoCoPlaygroundArgs):
  class CheetahRunArgs (line 369) | class CheetahRunArgs(MuJoCoPlaygroundArgs):
  class Go1JoystickFlatTerrainArgs (line 381) | class Go1JoystickFlatTerrainArgs(MuJoCoPlaygroundArgs):
  class Go1JoystickRoughTerrainArgs (line 391) | class Go1JoystickRoughTerrainArgs(MuJoCoPlaygroundArgs):
  class Go1GetupArgs (line 401) | class Go1GetupArgs(MuJoCoPlaygroundArgs):
  class LeapCubeReorientArgs (line 411) | class LeapCubeReorientArgs(MuJoCoPlaygroundArgs):
  class LeapCubeRotateZAxisArgs (line 422) | class LeapCubeRotateZAxisArgs(MuJoCoPlaygroundArgs):
  class IsaacLabArgs (line 433) | class IsaacLabArgs(BaseArgs):
  class IsaacLiftCubeFrankaArgs (line 447) | class IsaacLiftCubeFrankaArgs(IsaacLabArgs):
  class IsaacOpenDrawerFrankaArgs (line 464) | class IsaacOpenDrawerFrankaArgs(IsaacLabArgs):
  class IsaacVelocityFlatH1Args (line 475) | class IsaacVelocityFlatH1Args(IsaacLabArgs):
  class IsaacVelocityFlatG1Args (line 483) | class IsaacVelocityFlatG1Args(IsaacLabArgs):
  class IsaacVelocityRoughH1Args (line 491) | class IsaacVelocityRoughH1Args(IsaacLabArgs):
  class IsaacVelocityRoughG1Args (line 500) | class IsaacVelocityRoughG1Args(IsaacLabArgs):
  class IsaacReposeCubeAllegroDirectArgs (line 509) | class IsaacReposeCubeAllegroDirectArgs(IsaacLabArgs):
  class IsaacReposeCubeShadowDirectArgs (line 517) | class IsaacReposeCubeShadowDirectArgs(IsaacLabArgs):

FILE: fast_td3/train.py
  function main (line 53) | def main():

FILE: fast_td3/train_multigpu.py
  function setup_distributed (line 57) | def setup_distributed(rank: int, world_size: int):
  function main (line 72) | def main(rank: int, world_size: int):
Condensed preview — 25 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (792K chars).
[
  {
    "path": ".gitignore",
    "chars": 57,
    "preview": "models\nwandb\n*.pyc\n.ipynb_checkpoints\nfast_td3.egg-info/\n"
  },
  {
    "path": ".pre-commit-config.yaml",
    "chars": 91,
    "preview": "repos:\n-   repo: https://github.com/psf/black\n    rev: stable\n    hooks:\n    -   id: black "
  },
  {
    "path": "LICENSE",
    "chars": 16079,
    "preview": "This software is part of the BAIR Commons HIC Repository as of calendar year 2025.\n\n------------------------------------"
  },
  {
    "path": "README.md",
    "chars": 16938,
    "preview": "# FastTD3 - Simple and Fast RL for Humanoid Control\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.sv"
  },
  {
    "path": "data/humanoidbench_result.json",
    "chars": 132177,
    "preview": "{\n  \"h1hand_walk\": {\n    \"time\": [\n      0.0,\n      327.4388405844,\n      654.8776811688,\n      982.3165217531999,\n     "
  },
  {
    "path": "data/isaaclab_result.json",
    "chars": 380742,
    "preview": "{\n  \"Isaac-Velocity-Flat-G1-v0\": {\n    \"FastTD3\": {\n      \"time\": [\n        0.0,\n        426.030303030303,\n        852.0"
  },
  {
    "path": "data/playground_result.json",
    "chars": 16901,
    "preview": "{\n  \"G1JoystickFlatTerrain\": {\n    \"PPO\": {\n      \"time\": [\n        0.0,\n        213.46350627565,\n        426.9270125513"
  },
  {
    "path": "fast_td3/__init__.py",
    "chars": 585,
    "preview": "\"\"\"\nFast TD3 is a high-performance implementation of Twin Delayed Deep Deterministic Policy Gradient (TD3)\nwith distribu"
  },
  {
    "path": "fast_td3/environments/humanoid_bench_env.py",
    "chars": 4063,
    "preview": "from __future__ import annotations\n\nimport gymnasium as gym\n\nimport humanoid_bench\nfrom gymnasium.wrappers import TimeLi"
  },
  {
    "path": "fast_td3/environments/isaaclab_env.py",
    "chars": 3131,
    "preview": "from typing import Optional\n\nimport gymnasium as gym\nimport torch\n\n\nclass IsaacLabEnv:\n    \"\"\"Wrapper for IsaacLab envir"
  },
  {
    "path": "fast_td3/environments/mtbench_env.py",
    "chars": 5265,
    "preview": "from __future__ import annotations\n\nimport torch\nfrom omegaconf import OmegaConf\n\nimport isaacgym\nimport isaacgymenvs\n\n\n"
  },
  {
    "path": "fast_td3/environments/mujoco_playground_env.py",
    "chars": 5378,
    "preview": "from mujoco_playground import registry\nfrom mujoco_playground import wrapper_torch\n\nimport jax\nimport mujoco\n\n\nclass Pla"
  },
  {
    "path": "fast_td3/fast_td3.py",
    "chars": 9156,
    "preview": "import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\n\nclass DistributionalQNetwork(nn.Module):\n    def __"
  },
  {
    "path": "fast_td3/fast_td3_deploy.py",
    "chars": 3339,
    "preview": "import math\n\nimport torch\nimport torch.nn as nn\nfrom .fast_td3_utils import EmpiricalNormalization\nfrom .fast_td3 import"
  },
  {
    "path": "fast_td3/fast_td3_simbav2.py",
    "chars": 17126,
    "preview": "import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport math\n\n\ndef l2normalize(\n    tensor: torch.Tens"
  },
  {
    "path": "fast_td3/fast_td3_utils.py",
    "chars": 33023,
    "preview": "import os\n\nfrom typing import Optional\n\nimport torch\nimport torch.nn as nn\nimport torch.distributed as dist\n\nfrom tensor"
  },
  {
    "path": "fast_td3/hyperparams.py",
    "chars": 16876,
    "preview": "import os\nfrom dataclasses import dataclass\nimport tyro\n\n\n@dataclass\nclass BaseArgs:\n    # Default hyperparameters -- sp"
  },
  {
    "path": "fast_td3/train.py",
    "chars": 27613,
    "preview": "import os\nimport sys\n\nos.environ[\"TORCHDYNAMO_INLINE_INBUILT_NN_MODULES\"] = \"1\"\nos.environ[\"OMP_NUM_THREADS\"] = \"1\"\nif s"
  },
  {
    "path": "fast_td3/train_multigpu.py",
    "chars": 30521,
    "preview": "import os\nimport sys\n\nos.environ[\"TORCHDYNAMO_INLINE_INBUILT_NN_MODULES\"] = \"1\"\nos.environ[\"OMP_NUM_THREADS\"] = \"1\"\nif s"
  },
  {
    "path": "fast_td3/training_notebook.ipynb",
    "chars": 30268,
    "preview": "{\n \"cells\": [\n  {\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"# FastTD3 Training Notebook\\n\",\n   "
  },
  {
    "path": "requirements/requirements.txt",
    "chars": 417,
    "preview": "gymnasium<1.0.0\njax-jumpy==1.0.0 ; python_version >= \"3.8\" and python_version < \"3.11\"\nmatplotlib\nmoviepy\nnumpy<2.0\npand"
  },
  {
    "path": "requirements/requirements_isaacgym.txt",
    "chars": 212,
    "preview": "gymnasium<1.0.0\njax-jumpy==1.0.0 ; python_version >= \"3.8\" and python_version < \"3.11\"\nmatplotlib\nmoviepy\nnumpy<2.0\npand"
  },
  {
    "path": "requirements/requirements_playground.txt",
    "chars": 920,
    "preview": "gymnasium<1.0.0\njax-jumpy==1.0.0 ; python_version >= \"3.8\" and python_version < \"3.11\"\nmatplotlib\nmoviepy\nnumpy<2.0\npand"
  },
  {
    "path": "setup.py",
    "chars": 216,
    "preview": "from setuptools import setup, find_packages\n\nsetup(\n    name=\"fast_td3\",\n    version=\"0.1.0\",\n    description=\"FastTD3 i"
  },
  {
    "path": "sim2real.md",
    "chars": 4924,
    "preview": "# Guide for Sim2Real Training & Deployment\n\nThis guide provides guide to run sim-to-real experiments using FastTD3 and B"
  }
]

About this extraction

This page contains the full source code of the younggyoseo/FastTD3 GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 25 files (738.3 KB), approximately 254.1k tokens, and a symbol index with 162 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.

Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.

Copied to clipboard!